Hacker News new | past | comments | ask | show | jobs | submit login
Building a roam-like, networked, heavily-customized realtime editor, part 1 (namiwang.github.io)
126 points by namiheike on Nov 22, 2020 | hide | past | favorite | 34 comments



Absolutely love the sheer amount of developers throwing themselves at this problem. I've built a throwaway one for myself but don't really have a use for it, but I'm excited to look back in like a year or two and see what best practices people have landed on. Maybe we'll just end up with Mediawiki, dunno, but excited.


MediaWiki works fine with SQLite. If that's still too heavy, there's always DokuWiki. Love it.


I remember setting out to write a Workflowy/Roam style outline viewed editor using prosemirror and it wasn’t as easy. Kudos to the author for building one.

Also, why is that all these note-taking apps see everything as a list item? As for me, most of my writings are loosely structured scattered across paragraphs, quotes and codeblocks. Why isn’t this mainstream? Why is “Everything is a list item” winning?


You might like https://logseq.com/. It works with either markdown or emacs org-mode syntax. It also syncs with github, so you could do local editing too.


I have gone completely over to this from emacs recently. I can fully commit knowing if things don't go well it's back to emacs with no issues.


Look up Obsidian (https://obsidian.md/). Built by the team that built Dynalist. It's basically Markdown + wikilinks ([[like this]]), allowing you paragraphs, quotes, and codeblocks.

I found out that I end up using lists more often than not, but I like the flexibility it offers.


Been a paying user of Dynalist but instead of moving to Obsidian decided to give https://logseq.com/ a try and I love it so far! Everything is pushed to my private Git repo.


I‘d recommend Zettlr instead: open source, free, local (so you own your data), great active community.

Obsidian is closed source, and I’d say its features do not justify choosing that over Zettlr (unlike something like Notion.so)

https://www.zettlr.com/


We’re actually building the desktop app and local storage is almost done for the web app over at https://logseq.com ! We’re using the new Filesystem API so look forward to many of these tools supporting local storage in the future!


Thanks for the heads up, I’ll be sure to check your app out!

Your approach is also kind of similar to Joplin (joplinapp.org), another open source, privacy-focused, and cross-device app, but you have more features.


> Obsidian is closed source

Should be noted that the Core-Application is closed Source. The plugins are at the moment open source, or at least source available. I think the long term goal is to build on a rich plugin-ecosystem with a sleak core.

Also closed source doesn't really matter in this case, because it's build on open data-formats and running local. There is no vendor lockin. If the app disappears some day, you can still continue to use it or use a different markdown-editor (which are already existing today).


It’s appealing to impose some structure. You can still write paragraphs, one for each list item.

I like to write related paragraphs as a nested bullet item. It helps scanning the content later.


I wish more of these systems allowed arbitrary indentation as a feature instead of just providing list items. I've gotten used to this in OneNote, and of course in plain text and Python. I don't want an outliner necessarily, but want to be able to indent a group of paragraphs and images with further indented sub-points, then outdent to move on to the next point.

Of course a lot of these systems don't support it because MarkDown doesn't support it. An indented block is a code block and that's it.

I started putting together a Tiptap editor project, which itself is based on ProseMirror, but got overloaded. I also tried building something with CKEditor but also ran into too much personal friction trying to get a simple project working. I have to learn Node and JS at the same time and it's a lot.


Right! On one hand going deeper and back out of a though feels so much more natural, just like our brain branches off and returns back. But we are so used to paragraph based text due to its history and it feels more familiar. I like how in Notion you can indent and deindent normal paragraphs but that still doesn’t feel totally fluid, I’m not sure what it is. Maybe different modes or views for each is closer to the ideal?


I've thought about this for awhile, and I can only come up with two ideas: a) laziness or b) lack of imagination.

It's a bit of a harsh criticism, but we are spatial creatures who live in a world full of objects that are organized into different forms. The "list" is a very valuable organizational structure and convenient to implement digitally, but it is a rather simple structure, being a 1D-structure. We use "lists" (in the form of shelves and stacks) to store objects and give the collection order, but in order to interact with objects, we remove them from their list and interact with them in higher dimensions (spreading out on a desk, pinning to a wall, arranging in space, etc). The fact that we only have 1D structures in most software UIs may well be that higher-order structures are too complex to practically implement or are not viewed as necessary, but again, this is an argument out of "laziness", convenience to the developer who is implementing the UI. Or, designers and developers genuinely believe that we are most-suited to composing with 1D structures (eye-roll).


Interesting point, I do really enjoy the spatial aspect of graphs being incorporated into many knowledge tools now. I guess the valuable insight is that if 1D is bad and unwieldy, what’s a good alternative? Like a bulletin board?


I am writing my own kind of Roam-style editor which is more free-form. From my experience, it's way easier – from a technical perspective – to think in terms of list items (or "blocks" in general) where each one is some kind of mini-editor. It's much easier to link to a block instead of an arbitrary section of your document. Also, it's much easier to render a document consisting of blocks without thinking too much about performance.


It takes some getting used to, but it does make a certain amount of sense, in allowing for very easy refactoring / restructuring, and fine-grained control over the shape of the graph.


The world's largest "No Code" userbase can likely be found with spreadsheets. There's a stable OSS project which combines a spreadsheet user interface with a fast 2D layout engine for text and graphics. Desktop cross-platform on Linux, Win & Mac via wxWidgets: http://strlen.com/treesheets/

The cross-linking patriarch of this category, Lotus Agenda, can still be run via DOS emulation: https://lock.cmpxchg8b.com/lotusagenda.html


see also http://outliners.com to get the old More outliner and https://welcometosherwood.wordpress.com/2009/10/10/grandview... that presents grandview.

And tinderbox I guess: http://www.eastgate.com/Tinderbox/


The good old days when you were still getting a proper paper user guid when you bought expensive software. Sometimes missing those days.


My favourite in this space is https://tiddlyroam.org/, very usable, easily extended and cross-platform.


Athens looks promising, too.

https://github.com/athensresearch/athens/

https://www.notion.so/Athens-67e1c6068cb449ff935d10e882fd9b0...

While mighty impressive, looking at the tech stack[1] for both Athens and Roam I can't help thinking this is too much to support my note taking (if I want to self host).

[1] https://www.notion.so/Athens-67e1c6068cb449ff935d10e882fd9b0...


Did you try athens locally? First time I hear about them but seems damn interesting.


> Did you try athens locally?

Did not, just followed the development. It seems like an extremely ambitious undertaking.


Slate is a great block based editor. The downside and the reason I didn’t choose it for notebrook [0] is because its contenteditable implementation doesn’t work on Android. You may want to double check that with ProseMirror. Please feel free to reach out to me (email in profile) to discuss your project further, as I’ve had some experience building an offline-first, reactive tag-based note app, and best of luck!

[0]: https://notebrook.com


There is a fully funded Kickstarter to bring support to Android [0]. We’re using Slate at my startup [1] and have been really happy with it. Our editor is pretty fancy and is relatively easy to maintain, test etc.

0: https://www.kickstarter.com/projects/sunnyhirai/add-android-...

1: https://kitemaker.co, a super fast issue tracker/product management tool with deep integrations to your team’s favorite tools


Thanks for the update, great news! I really love Slate but the Android support was a non-starter for me, and I ended up using React-Quill. It has its own share of issues, but at least the native editing was decent on Android. I talked with Sunny about working on the Android port on Slate's GitHub earlier in the year, and spent a few days working to add support myself based on his earlier work, but it was not what I wanted to focus on for my application at the time and I decided not to pursue Slate further. It's very impressive they can get the sponsorship to do this and it makes me really excited for Slate's future.


This is really interesting, can't wait for more.

Personally I cannot really get the hang of linked editors like this that only use links or basic tags for structure. They require too much upkeep to keep things organized. I'm working on my own solution that goes a different way, but it's nice to see more open source implementations of these types of editors.


Neat! I enjoyed your thoughts about choosing a data structure and editor component. I'm also building something in this space (https://thinktool.io/, https://github.com/c2d7fa/thinktool), and I decided on a somewhat different data structure, with pros and cons.

Instead of having pages that contain blocks inside of them (like Roam), I decided to have just one type of item. These items can have other items as children and parents, and they also contain text content which can link (bidirectionally) to other items.

The main advantage of this approach is that items can have multiple parents. So in practice, you never have to think about whether a note should be its own page or just a block inside a different page. This just feels more elegant than Roam's approach to me.

The main disadvantage is that you're now working with a graph rather than a tree. You can end up with funky situations like an item that's its own parent. Since I still want the user to interact with the app through a tree-based UI, I have to add an intermediary data structure representing what the user can actually see. This data structure needs to be built lazily as the user expands and collapses items, so we don't try to represent an infinite loop.

Whenever the user does something (adds an item, edits it, creates a link, etc.), we need to update both the intermediary data structure and the persistent data -- and ensure that these changes are kept in sync. For example, adding a link is a simple change in the graph, but it may require multiple updates to the tree.[1]

I also ended up going with ProseMirror for the editor component (after trying Quill, Slate, an input-based approach and using contenteditable directly). I'm really happy with ProseMirror, even though integrating it wasn't entirely pain-free.[2]

---

[1] You can see the code for the tree-representation here if you're interested, although it's not necessarily written for public consumption: https://github.com/c2d7fa/thinktool/blob/master/src/client/s...

[2] The editor code is here: https://github.com/c2d7fa/thinktool/blob/master/src/client/s... -- same disclaimer as above.


cool. glad to see the progress.


What does roam mean in this context? When I read roam editor I thought ROAM terrain algorithm.



Roam is a new outliner out there which created some fizz a couple of months ago.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: