Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

node.js Workers + SharedArrayBuffers?

I'm not sure how well supported they are / haven't used them, but I think they enable this?



Yes that allows for shared memory. But the TypeScript compiler is written in TypeScript, and you can't put JavaScript objects (such as the TypeScript AST) in a SharedArrayBuffer. So you'd have to port the TypeScript compiler to another language that can target multithreaded WASM to do this.

You could also try sending JS objects between workers instead of using SharedArrayBuffer. But that copies the memory, which can take a significant amount of time, and of course also excludes certain kinds of advanced parallelism that require shared memory (instead of copied memory). Another drawback of using the JS worker model like this is that having multiple JS VMs probably means fewer CPU cores available due to there being multiple independent garbage collectors. I observed an effect like this when I tried to parallelize esbuild's JavaScript plugins. Running them in separate workers seemed to cause the gains from parallelism to stop when there were half as many workers as CPUs. I'm guessing is because half of the CPUs are busy collecting garbage for the other half. Using a language like Go with real shared memory presumably makes it easier for fewer resources to be spent on garbage collection.


Folks at Parcel wrote serializers that uses SharedArrayBuffer for their core stuff like graph[1]. I agree with parent comment that this didn't have to happen in Go, JS would've been fine if they were willing to depart from the source enough.

[1] https://github.com/parcel-bundler/parcel/tree/v2/packages/co...




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: