Hacker News new | past | comments | ask | show | jobs | submit login

Hi - Congratulations! In the streaming use case, does it restart from where it left off in case the target peer or source peer is down/restarts etc?



Great question. Yes it does. PeerDB keeps track of what rows have been streamed and what are yet to be streamed. During failures (restarts, crashes etc), it uses this to resume from where it left off. More details on how we do it can be found in this blog - https://blog.peerdb.io/using-temporal-to-scale-data-synchron...


Thanks so if we do a join on two large tables, does it wait for the query to complete or it can start straight away.

Or the reverse, is it possible to have a forever running query that can stream results as and when new data comes...


Yes, PeerDB can stream query data continuously. You need to specify a watermark column (incremental id or timestamp column) as a part of the mirror. PeerDB uses this column to keep track of data that needs to be synced.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: