> As for the benchmark, who writes their own load balancer? Isn't this generally a solved problem?
You'd be surprised, but no, it isn't a solved problem because the problem can't be solved in a general way and a lot of people are writing their own load balancers.
> If the point is to extract max performance from a simple ping-pong server then I'd go right to C and epoll/libevent/etc.
For a ping-pong server, sure, but a ping-pong server has no value in real life. The point of building your own load balancer is for doing custom load balancing and routing, based on individual needs of the project.
The complexity grows exponentially and while big companies are building such things in C, mere mortals do not have the resources for it. And in fact building on top of libevent is not going to give you performance advantages over Java NIO. The only thing that sucks about building on top of the JVM (compared to C) is the garbage collector, that can lead to unpredictable latencies related to blocking the whole process during the marking phase, even with CMS, although it is manageable and much better productivity-wise than to deal with GLibc incompatibilities or with multi-threading in C, plus I hear that G1 from JDK7 for large heaps and CPUs with multiple cores, or Azul's pauseless GC are awesome.
There's always a tradeoff of course, but sometimes the best path for many projects is the solution that makes the best compromises between productivity and performance and that's why I'm a happy Scala user and love the JVM.
I'm always amused by the slight fear that web developers treat C with (and even more so by comments like "does anyone actually write anything in C/C++ any more?"[1]), although I do appreciate the compliment that us embedded software engineers aren't "mere mortals" ;-)
Sure, even for people who are experienced with it it takes a bit longer to write robust code in C than it does in a higher level language, but the performance benefits can be enormous. If you've got a well defined block of functionality that needs to be quick, you might well be better off taking the time to do a C implementation than you would be spending ages optimising a Scala implementation.
Of course I spend a lot of my time arguing the opposite at work - we write a lot of C++ out of habit, but it's not performance critical at all and it's running on Linux on an Intel Atom. We'd be a lot more productive writing the app in (for example) Python.
[1] Short version, yes, lot's more than in any single language you use (cheating slightly here by counting C and C++ together). Go a few levels down in the stack of whatever you're writing and you'll get to C/C++ probably quite quickly, or worst case at the OS (and if you don't have an OS, then you're writing embedded code, and you started in C).
I'm not a web developer and I'm not scared of C, but I very frequently find that optimizing the JVM is much easier than writing a similar piece of code in C for my real world problems.
The short answer for me is all the great tooling that comes with a JVM solution as well as the simplicity of deployment. Getting a C/C++ native solution to play nice with a continuous integration environment is often very painful (especially compared to a system that is JVM only). Unit testing tools are better for the JVM, IDE/Debugger/Profiling support is better, etc.
All of these things (and the fact that good JVM code can be very close to C/C++ performance) means I don't roll with C because it's scary, rather because it's a pain.
> I do appreciate the compliment that us embedded software engineers aren't "mere mortals" ;-)
With all due respect to embedded software, the stuff that you have to deal with is limited in scope. I'm not saying it isn't hard or challenging, but nonetheless it is limited in scope. And this matters - I also worked a lot with C/C++ and while C/C++ developers can easily reason about things like cache locality or branch predictions or data structures, if you want to block them at an interview the easiest thing to do is to ask them to do some strings processing. The point of having high-level abstractions is to build bigger, more complex things and with C/C++ the pain starts straight from the get-go.
> even for people who are experienced with it it takes a bit longer to write robust code in C than it does in a higher level language, but the performance benefits can be enormous
Err, no, not really. Tackling multi-threading issues in C is absolutely horrible, coupled with the fact that C/C++ do not have their own memory model, so you end up with really hard to reproduce bugs by just updating some packages on the host OS. Libevent has always been a joy to work with in the context of multi-threading, the result being a whole generation of insecure and freakishly sensitive servers.
On top of the JVM you've got a good memory model, you've got multi-threading done right, you've got async I/O without headaches and as far as the concurrency model is concerned, you can take your pick from light-weight actors, futures/promises, shared transactional memory, parallel collections, map/reduce and so on, without any of the associated headaches.
There's also the issue of using all the hardware that you have. We are only scratching the surface of what can be done in terms of GPUs, but a high-level language running on top of the JVM or .NET makes it feasible to use the available GPUs or other available hardware resources by using high-level DSLs. The Liszt DSL is a really cool example of what I'm talking about: http://liszt.stanford.edu/
So if you mean "performance" as in measuring the time it takes for a while-loop to finish, then sure C is the best, but go further than that to efficient usage of all available resources and the problem is not really black and white.
> you would be spending ages optimising a Scala implementation
That's not true in my experience. At the very least with the JVM you can quickly and easily attach a profiler to any live production instance. In so far as profiling and debugging tools are concerned, the JVM is the best. And that's where the real difference comes from my experience in the real world - optimizing without real profiling is premature optimization and low level optimizations many times stay in the way of more high-level architectural optimizations with better benefits.
Speaking of spending ages on stuff and the server-side, good luck debugging what happened from a 3 GB core dump generated on a segfault, because somebody though that an Int is actually a Long.
Also, just lust week I tested our servers on top of Amazon's c1.xlarge instances, which have to be 64bits, servers which are normally running on c1.medium which are very cost efficient, but that are better left to run with a 32bits OS. In 15 minutes I was able to basically migrate from 32bits to 64bits. Good luck doing that with C/C++.
You'd be surprised, but no, it isn't a solved problem because the problem can't be solved in a general way and a lot of people are writing their own load balancers.
> If the point is to extract max performance from a simple ping-pong server then I'd go right to C and epoll/libevent/etc.
For a ping-pong server, sure, but a ping-pong server has no value in real life. The point of building your own load balancer is for doing custom load balancing and routing, based on individual needs of the project.
The complexity grows exponentially and while big companies are building such things in C, mere mortals do not have the resources for it. And in fact building on top of libevent is not going to give you performance advantages over Java NIO. The only thing that sucks about building on top of the JVM (compared to C) is the garbage collector, that can lead to unpredictable latencies related to blocking the whole process during the marking phase, even with CMS, although it is manageable and much better productivity-wise than to deal with GLibc incompatibilities or with multi-threading in C, plus I hear that G1 from JDK7 for large heaps and CPUs with multiple cores, or Azul's pauseless GC are awesome.
There's always a tradeoff of course, but sometimes the best path for many projects is the solution that makes the best compromises between productivity and performance and that's why I'm a happy Scala user and love the JVM.