Hacker News new | past | comments | ask | show | jobs | submit login

I generally agree with the sentiment about scaling across cores and nodes being the same; after all, you're eventually going to need to scale across nodes so you may as well go ahead and get that working.

But there are times when it can be a pain. For instance, a node app I run at scale maintains an in-memory read cache that rarely gets updated. When we scale with processes, we end up having to duplicate the cache across every process. We could set up a helper process and have it manage the cache over a socket, or possibly write a c++ module that shares some memory but at that point it's no longer simple and/or there are more parts to potentially fail.




Or... just use the battle-tested tool that solves this exact problem well for you. Put your cache into Redis or other in-memory cache.


A network-addressable store such as Redis is not the same as just being able to pull stuff out of memory. A map guarded by an RWMutex is simpler in code, probably more performant for simple access patterns, simpler operationally and effectively can't fail, as the parent points out.

Yes, Redis can be the right tool. But, note how quickly we went from "I just want to run on multiple cores" to "I need to share data between my cluster workers so let's install Redis". It's just a total blowup of moving parts.

Great-grandparent argument reeks of elitism. I know how to deal with concurrency, parallelism, avoiding data races. Doesn't mean I need to waste my time implementing the primitives every project. To say that the language forcing you into multi-process architecture is a benefit, is a pitifully desperate remark. I wonder what single digit percentage of projects actually go into that territory? Maybe even less than 1%? Maybe with a runtime suitable for the task (e.g. Go or Java or hand-woven C++), less than 0.1%, because you do not run into pointless runtime bottlenecks to begin with?

I have implemented such projects, because we needed the workers to be geographically distributed, not because the language runtime fucked us over. This was a decision made in DESIGN not in DEFECT REMEDIATION.

I can think of a handful of "single-purposed" reasons to use Node on the server, such as DOM rendering (and other stuff that is only available in Javascript-land). But a serious project, no way. Not sure how people can stomach the inherent lack of safety in the language, the huge (relative to comparable substitutes) number of limitations in the runtime and the ecosystem churn.


Yep, redis would solve this. Shareable cache across a distributed cluster of node.js instances.


... and now you have two problems?


"helper process and have it manage the cache over a socket" is essentially redis, and another thing that introduces latency and can fail.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: