Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I had an interest in CRDTs for quite a while now, as I like the local first philosophy of developing software, that works offline in its entirely but can also work online, a sort of progressive enhancement [0]. Recently I've been looking into Loro [1] which seems like it is able to effectively merge disparate offline text edits together, by using some new algorithms that were written about last year, such as the Event Graph Walker [2]. I've been combining this with ElectricSQL [3], which is a sync engine for Postgres. In the past they had their own CRDT right inside Postgres which would then sync tables, but they have rewritten their product to focus primarily on being a sync engine first and perhaps a conflict resolution library second. Therefore, I store the Loro changes binary in a table in Postgres as a blob that I then sync via Electric to all my clients.

Ultimately though, it is as you and others like @zamalek in this thread have said, the mathematical notion of conflict resolution might not actually mean anything semantically, therefore it is difficult to have long running offline edits merge together cohesively; it works with things like Google Docs where the user can see what other users have written in real time, which works for 99% of use cases, and sometimes I wonder whether one really needs such long running offline syncs, as it seems to be a very niche goal to target. Short running offline is nice to have and even necessary, especially for products one expects to work wholly offline, but it is the long term I don't see much value in, as in not collaborating online for weeks or months at a time but still expecting cohesive edit merges.

[0] https://localfirstweb.dev/

[1] https://loro.dev/

[2] https://loro.dev/docs/advanced/event_graph_walker

[3] https://electric-sql.com/



Alas, Loro fails The Color of Pomegranates test as well. (JSON trace, really cool toy they got there: https://pastebin.com/6dSDc6Su)


Yes, it is as I mentioned in my 2nd paragraph, mathematical conflict resolution is not semantically relevant for humans much of the time. That is why I don't think automated merge conflict resolution with no human input can really work, that is why Git asks you to resolve merge conflicts before committing again. CRDTs can only really help when users edit disparate pieces of data, and if editing the same piece of data, some set of users need to be there, as in an online rather than offline capacity, to facilitate the merges, such as in Google Docs where if I edit a sentence with a spelling correction and you delete the sentence entirely, I will ask you directly what's up.

Now, what some Git merge software is doing is using LLMs to coordinate merges, which I think might be useful to do so in plain text editing too, perhaps embedded into these CRDT libraries, but fundamentally, there must be someone to facilitate merges, whether it be a human agent or an AI one. It is impossible to do so with an entirely automated solution simply because the machine does not know the intents of all its users.


The text sync algorithm I wrote succeeds at the "The Color of Pomegranates" test. See https://cocalc.com/wstein/dev/share/files/2024-12-06.md for some details of the exact patches. This algorithm, which we came up with in 2015, is described at https://blog.cocalc.com/2018/10/11/collaborative-editing.htm... It's a different approach than that taken by Yjs, Automerge, and Egwalker, with signficant pros (and cons!) relative to those algorithms. It has been used in production in CoCalc for over 8 years, and was also used by the recently shutdown Noteable Jupyter notebook project.


This is absolutely fascinating and immediately one of my favorite articles of all time, full stop. Along the way, it implicitly illustrates a path to successful engineering x product thinking path that I've never seen written down somewhere and I think is crucial for sustainable impact. Thank you!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: