I've been coding in Node for a couple of years. I find it interesting. But I'm aware that the async model of Node inverts the flow of control, turning code inside out and makes application logic difficult to scrutinize and reason about. Stack traces in traditional multithreaded languages are easy to understand, whereas in Node a stack trace is necessarily filled with unrelated calls - or perhaps no stack at all in the case of next tick deferred callbacks - due to the single-threaded nature of the model. This necessitates passing around context via closures (or god forbid global variables!). The "context" in a threaded language is typically just the stack. With Node's async Promises and equivalents we can trick ourselves into simulating linear program flow, but it's a pale imitation of the real thing.
This got me thinking - why did the single-threaded "green" threading programming model like how the Java JVM was initially implemented all those years ago fall out of favor? It would be as performant as Node presently is but without the difficult async logic for the end user programmer. Async programming and green threads are duals - both use event queues behind the scenes to drive the scheduling. The complexity of the actual I/O and timer event callbacks is hidden from the programmer in green threads. The pseudo "thread" context switches would only be done upon blocking I/O. No need for actual mutexes as the program is still actually single threaded. Stack traces of green threads within a program would be easy to decipher. Computation on a green thread would also block all I/O as it would in Node, but we'd know that going in - no different from Node in that regard. Just the programming model would be simpler.
The programming model you are describing is almost exactly the model Go has, it just takes it a step further and allows N:M "green threading", so you can use all your cores.
It is exactly Go's model. Go, since the very beginning, has a N:M threading model. runtime.GOMAXPROCS() resp. $GOMAXPROCS is your friend to control the amount of OS threads, and since 1.5 or so, it is by default equivalent to the number of CPUs visible to the Go runtime, anyway.
Well, he was describing a N:1, not N:M: "No need for actual mutexes as the program is still actually single threaded", but yes, Go has been N:M from the start.
I would guess it was the immaturity of the java IO layer that didn't play well with green threads. You need a good asycn IO layer on all the platforms that java was supporting (solaris and windows, plus others) for the green threads to work well.
Node's javascript engine v8 can only run in one thread, if you want to access it from a different thread you need to take a lock (for example in Chrome / Chromium you need to take an isolate lock; each page runs it's own isolated v8 engine (read only code pages are shared, all other javascript pages (data plus read/write code pages) are private); in this sense it is very similar to the GIL in Python. So although node/v8 allows for concurrency it does not allow for multi-threaded parallel execution.[0]
Because of the way tasks are scheduled in node.js you can have head of line blocking; i.e. a task that is scheduled to run first can block all the subsequent tasks, the issue that the OP suffered from. You can sometimes reduce this by calling process.nextTick(), but you need to remember that it is only IO is non-blocking in node.js.[1]
Since Google changed the V8 API to mandate use of Isolate pointers passed in to V8 functions as a parameter rather than implicitly using thread-local data for engine storage it is now possible to run separate V8 instances within the same thread (one at a time), or from different threads as they do not share state. One of the many issues Node has to contend with in using the V8 engine is a perpetually changing API surface.
I don't think java threading fall out of favor. It is just that people jump to new shiny things without understanding what they are trading for. I mean 200/req per minute is not something high to start with --> 3req/sec is now days high load ?
Not really a pale imitation, works just as well. The async and await keywords could be shorter. Toffeescript and Livescript are more concise than the new ECMAScript with their operators for async but it's just a little more text to read.
