Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

100GB of RAM — I'm curious, why is this a web app? Sounds like an internal tool that could have been written in, for example, C#.


It's actually an internal build tool (forked from an open source tool) used to build our public applications and runs in Nodejs. We are rewriting it in Rust but it's a complex tool and that takes a lot of time.

In the mean time, a "build" using this Nodejs based tool takes about an hour and uses 100gb of RAM.

By contrast, preliminary unoptimized results of the Rust tool brings build times down from 1 hour to 1 minute and maxes out at 6gb of ram - but that'll be finished in 2 years.

It literally costs millions per year to run and is a nightmare to maintain.

To be fair, the JavaScript itself is actually surprisingly performant - it's the multi-threading that's the real problem.

It would be a non issue if there were memory sync primitives that could be sent between threads but because of how v8 isolates work, this isn't possible without changes to v8. So we are forced to copy gigantic graphs of data between threads and manually synchronize them with postMessages.

Running a build on the main JS thread (no workers) uses 10gb of ram and takes 5 hours. So shared memory would solve our issue and avoid the need to rewrite it.

Attempts to centralize the data (in a DB or similar) perform worse due to the de/serialization of the data. Other attempts of using a shared in-memory database shares the same issue. We also need to modify every shared structure in the codebase to use getters/setters with a copy on read approach (so memory blows up any way).

It's not that JavaScript itself is the wrong language, but that the isolated nature of the runtimes fail in multi-threaded use cases.


Microsoft doesn’t exist in the hive mind of Silicon Valley. That’s the only reason.

“We’d rather spend 3x the effort to bash things with open source rocks than ever use a proprietary hammer!”

“.NET is open source. It also comes with a steam roller.”

“We’ve made up our minds!”

PS: I just had to work on a Node.js app for the first time in my career. After dotnet core it feels like working with children’s wood blocks.


.NET being "open source" doesn't mean it's not proprietary - Microsoft still effectively controls its future.


Similar experience here. Though I moved from C# to Rust and it's become my favorite language of all time.

I'd write everything in it if I could, haha


I find the gaps in the Rust standard library annoying. The batteries-not-included design just doesn't do it for me, especially when it's for commonly used things such as the guid type and common interfaces.


That's fair. The std lib is the biggest pain point for me also but it's easy enough to work around. That and the poor ergonomics of async/await/futures




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: