Not the guy who posted above but I think I have relevant experience. After going through different options for message queuing we have decided that using S3 and object will be our best option for many reasons. Some of those reasons:
- durable
- cheap
- scalable
- we do not need to read messages in a strict order
- using S3's API is super easy
- no operations overhead
- no compute resources (for the queue part)
After doing it some time I think that I was drinking too much cool-aid with streaming services like Kinesis and Kafka and failed to see how easy it is to implement a simple analytics even service using purely S3.
- durable
- cheap
- scalable
- we do not need to read messages in a strict order
- using S3's API is super easy
- no operations overhead
- no compute resources (for the queue part)
After doing it some time I think that I was drinking too much cool-aid with streaming services like Kinesis and Kafka and failed to see how easy it is to implement a simple analytics even service using purely S3.