Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Why not mention the architecture that comes closest to exactly-once delivery? If you store a Kafka offset along with your application state in a transactional datastore, then for all intents and purposes you have exactly-once delivery semantics. This is something I really like about Kafka’s design.


This is again assuming that you have no side effects. Imagine that you want to email users based on a list in Kafka. You read the offset in a transaction and update it. But do you send the email inside the transaction or after closing it? You are back to picking between at-least-once and at-most-once.


Just put the offset in the email to users and make them keep track of it??


Move the problem to another system does not solve the problem. Someone is necessarily dealing with it.


Exactly - the user could die after opening the email but before reading the number!


You got some good jokes as answers already, but what you do if you care about correctness is to store a “send email job” in your transactional data store.


What transactional data store capable of sending email do you recommend?


Just about any transactional datastore will do. Just google for “job queue in [insert name of datastore here]”.


You've missed the point: How do you tie sending email to the transactional semantics? Your email server and your SQL server (or whatever) make just as much of a distributed system as Kafka and the email server without some other store in the middle. If you want to solve it really, your email server itself needs to participate in the consensus process in some way.


I suspect it's not mentioned because in the real world there are ways to work around this limitation. And because those work arounds actually work, there's nothing interesting to say. The post is much ado about nothing.


Yes, it’s always possible to build a big pile of workarounds. I agree.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: