This is not clear. Multicore and distributed computing are unrelated at best.
To make it clearer, can you explain what you are doing in erlang that you couldn't do in ocaml?
I am message passing thousands of financial ticks per second to hundreds of terminal-user consumers, each of which may wish to preselect filters that they wish applied to each series, or combinations of series. This is trivially easy to organize in Erlang/OTP, it is trivially easy to scale, and I have no clue how I would organize it in Ocaml without going to message passing first principles, and in which case I would have to build the entire load balancing, PID and security infrastructure from scratch. Moreover I do not see how you do not see that multicore and multi-node are related. On Erlang, having two 8-core boxes is almost the same as having one 16-core box. Again, if there is something I am missing about the Ocaml environment I would be happy to know about it.
It sounds the real benefit for you is in OTP rather than anything intrinsic to Erlang's multicore implementation.
> not see that multicore and multi-node are related
There is a huge distinction in implementation, which Erlang/OTP abstracts over for you somewhat (but not entirely...). By optimizing for the distributed (multi-node) scenario, you decouple the that work from the language implementation -- and multiple processes per CPU would benefit just as well as a cluster. A distributed cluster does not benefit from a multi-core implementation. Given the baggage most languages have with the GIL (including OCaml), optimizing for single threaded performance seems like the correct decision here.
With Erlang, distribution is accomplished with the epmd registry, something similar could be done with OCaml as well. As you noted, there is no OTP-like system for OCaml, but the absence of one does not hinge on any multicore implementation. Single-threaded OCaml processes handle concurrency just fine, and usually faster than Erlang, to boot.
Personally, if the OCaml multicore work ends up slowing down the core runtime appreciably, I'm going to be upset.
> Personally, if the OCaml multicore work ends up slowing down the core runtime appreciably, I'm going to be upset
That's what I meant in my original post about multicore seeming controversial. That's part of the reason that for me, a super-speedy single-core Ocaml coupled with an industrial strength OTP-style distributed computing framework, would be awesome. I'm actually not that excited about multicore. My use case is lots of streaming data coming in, needing to be parsed, lots of heavy lifting stats work done in real time, then distributed to clients. So it needs both very high performance on the single thread (currently using GPU-accelerated calls to Numpy), and then very efficiently be able to distribute heterogenous data to lots of endpoints. I really would love to be able to do it all in Ocaml.
You may be interested in Cloud Haskell. It's basically Erlang networking nicely re-implemented in Haskell with the strong static typing you expect from Haskell. Gets you some of the advantages of both erlang and ocaml.
For what it's worth, "Cloud Haskell", the project, corresponds to the distributed-process library, which may be easier to search for than cloud Haskell. The API is described in chapter 14 of Parallel and Concurrent Programming in Haskell, by Simon Marlow. (I haven't been paying close attention to the distributed-process library, so I don't know whether that chapter is up-to-date with recent versions of the library.)
Similarly, Scala with Akka gives you an ML-like language with first-class Linux support, excellent multicore performance, and Erlang-style distribution.
Regarding the multicore thing it had been answered already: if you have to run a distributed service in a 8 cores machine then just run 8 binaries. If you are using threads to write more elegant synchronous-like code in replacement of even driven asynchronous io then you do not need a concurrent runtime for that.
Now, regarding the availability of libraries to build distributed system in ocaml you are totally right. For years it seams everybody was happy with mpi, then we had some sort of map reduce framework, and that's it as far as I'm aware.
I would expect the growing popularity of mirage OS to improve this situation, though.
No, the point is that if you're going to do distributed computing then multicore is a waste of time - you might as well just run one process on each core and treat each core like a separate machine (where some machines happen to have a particularly low-latency/high-bandwidth network connection). You still need a good distributed runtime, but if you believe distributed is the future then multicore is a dead end.
