I had an interview where I was asked how I would guarantee that an event happened in addition to a database update (transactionally).
It sounded kind of impossible, I said as much, and then proposed a different approach. The interviewer persisted and claimed that it could be done with 'the outbox pattern'.
I disagreed and ended the interview there. Later when I was chatting about it with a former colleague, he said "Oh, they solved the two generals problem?"
> Every use I've seen sent events after database transactions, with the event not part of the transaction.
I don't quite see what the outbox pattern has to do with the two generals problem.
The point of the outbox pattern is that a durable record of the need to send an event is stored in the DB as part of the DB txn, taking advantage of ACID guarantees.
Once you have that durable record in your DB, you can essentially treat your DB as a queue (there's lot of great articles on how to do this with Postgres for instance) for some worker processes to later process the queue records, and send the events.
The worker processes in turn can decide if they want to attempt at least once, or at most once delivery of the message. Of course if you choose the later, then maybe your event is never sent, and perhaps that was the point you were trying to make to the interviewer.
They key takeaway though is that you are no longer reliant on the original process that stores the DB txn to also send the event, which can fail for any number of reasons, and may have no path to recovery. In other words, at least once delivery is now an option on the table.
> I don't quite see what the outbox pattern has to do with the two generals problem.
Well then, hopefully you would have found it an unsatisfactory 'solution' and walked away from that interview too ;)
> Once you have that durable record in your DB, you can essentially treat your DB as a queue (there's lot of great articles on how to do this with Postgres for instance) for some worker processes to later process the queue records, and send the events.
Yeah but I already have a queue I can treat as a queue. It's called Kafka.
The goal of the outbox pattern is at-least-once publishing though, not only-once. You either get P + (eventually) at least one copy of K, or you get no P and no K.
Without the outbox you can get P without K or K without P, which lead to consumers out of sync with the producer.
This requires the consumer to be able to deal with repeated events to some extent, but you usually want that anyway, since an event can be processed twice even if it appears only once in the topic. For instance if the consumer crashes after processing but before updating the offset.
> The goal of the outbox pattern is at-least-once publishing though, not only-once.
Right, which is why it's an unacceptable solution to 'transacting over postgres and Kafka', and why I wouldn't want to work for a company that wants me to believe differently.
I think you're just using "transaction" in a different sense than what the interviewer meant; "guarantee that an event happened in addition to a database update" sounds like at-least-once to me, and it's normally what you would want in this kind of system.
It sounded kind of impossible, I said as much, and then proposed a different approach. The interviewer persisted and claimed that it could be done with 'the outbox pattern'.
I disagreed and ended the interview there. Later when I was chatting about it with a former colleague, he said "Oh, they solved the two generals problem?"
> Every use I've seen sent events after database transactions, with the event not part of the transaction.
Maybe this is what they were doing.