I just can't resist, Serbs have saying 'almost is what makes girl unhappy' :).
On more serious note, ruby is definitely better when it comes to richness of available libraries which is what node is missing. I think in real life node can handle more, I heard that issues with memory leaking are solved, probably uses less memory as well.
I agree with conclusion that both are more then capable or handling async problems, I would use event machine on ruby end, this whole example seems flawed a bit. Anyhow, thanks for posting your results.
I would be interested to see how the unicorn variant, rainbows, would benchmark in this test. Rainbows seems much better suited for high concurrency applications because it utilizes fibers for concurrency as well as worker processes. http://rainbows.rubyforge.org/ Also after looking at the sinatra app's code, I believe all requests to MongoDB are going to be blocking. For the comparison to be more fair, the ruby app should be written to take advantage of Goliath, em-synchrony, event machine, or another non-blocking ruby fiber based option. Until the ruby app is non-blocking, this comparison is apples to oranges.
This is great work but...please redo the charts to start at Zero and end at 50,000 to keep things consistent and prevent Edward Tufte from killing a kitten if he ever sees your charts.
"The urge to contextualize the data is a good one, but context does not come from empty vertical space reaching down to zero, a number which does not even occur in a good many data sets. Instead, for context, show more data horizontally! "
A non-zero baseline is fine in some cases (line chart of the temperature of a person with a fever for example), but the issues here were multiple charts at varying Y scale starting points, and bar size comparisons of a property that we do want to go to zero (zero failures).
Yeah, the graphs are misleading because of the varying Y scale.
I do like the conclusions section, though. It presented some insightful analysis of dyno/worker efficiency
>It could be that I’m benchmarking from only 1 server but I’ve seen almost no difference between having 40 dynos or 60. You’ll see one when you receive the bill so be cautious, especially if you use an auto-scale tool.
>The same applies to node with cluster, you can do more with 15 dynos running cluster with 3 workers than with 60 dynos of node alone (for a quarter of the price!).
But node.js doesn't have any concurrency. It's single-process, single-threaded and event driven, so by definition it's not concurrent. It's just not blocking.
Also, was someone really trying to run performance benchmarks on a shared machine running on a VM and expecting to get anything meaningful out? Really? Like really? I'm embarrassed for you.
Let's not look at things like thread safety, resource consumption, average/best case/worst case latency, quality of available libraries and their support for concurrency, or code quality/maintainability issues.
Meh.
At least he provided the source code and plenty of data. That's cool.
I don't think we are required to provide a better analysis in order to point out the problems with a flawed one.
Whatever the author is measuring here, it's most certainly not "node" vs "ruby". This should be obvious when you look at the test-apps that he's using (which involve mongodb and serialization).
It's very easy to complain, cast stones, etc. I don't think I've seen a benchmark on HN or reddit where many of the comments were not smug complaints. Yet, I've never seen any of the complainers produce a benchmark with the exception of zedshaw and one other person whose name I've forgotten.
Just because someone posts a benchmark that is obviously flawed, doesn't mean that someone else should be obligated to do their own benchmark as a counter.
These guys didn't ask for node.js to be used in any benchmark.
The onus is on the person who chooses to do a benchmark, and publish the results, to get it right. If they don't get it right, then they are in the wrong and deserve the criticism that they get as a result.
Because it's like complaining about the weather. It doesn't do anything to make things better. I think there's too much negativity on the Internet already.
There is too much negativity on the Internet, but there's also too much of just about everything else. The ease of publishing on the Internet means that people don't think too critically about their work prior to publishing it.
For example, if these guys were publishing this benchmark as part of a study or argument in a quarterly magazine -- and this was there only opportunity to publish it for the next year or so -- then they would have been far more critical of their own work prior to publishing it. Instead they've quickly thrown together something and pushed it onto the Internet without too much further thought.
To some extent this is like a peer-review. If I submit a paper to a journal, should my response to all criticism be, "Well, if you think I didn't do the experiment right, then you do it!"
This is not a Ruby vs Node.js benchmark because Node.js is still a layer above the V8 Javascript engine. Moreover Node.js was optimize for things like concurrency and this is similar to what eventmachine is optimized for. The difference is node.js makes the event model a language construct. I think it would have been better to use eventmachine but you did not because:
> At first I tried to compare bare node.js against eventmachine_httpserver. It did quickly became obvious that this kind of micro-benchmark wasn’t going to be very helpful in deciding which one to choose.
Your first observation was the correct one. The best thing to do is just pick a language and go with the best tools available.
Yes the nodejs concurrency model takes some time getting used to, but working with threads is much harder imo. Raw speed benchmarks don't do enough to give a full picture.
This is comparing nodejs with Ruby's EventMachine, so where do threads enter the picture? It's been a while since I looked at EM, but if I remember correctly, it's pretty much in the same area as Python's Twisted or node.js, as opposed to something like plain ol' Java threads or Erlang.
I don't actually even see eventmachine in there, aside from a remark about having tried it in the opening introduction. Seems it is just comparing node with sinatra+mongomapper running under thin or unicorn.
Based on the title I assumed it was something like goliath or sinatra/async with rainbows!/thin. Looking at the git repo linked in the article though, I don't believe that is the case.
EventMachine uses an event loop like node, so threads are somewhat irrelevant here. Unless "workers" are threads and not actual processes, but I am pretty sure they are processes.
On more serious note, ruby is definitely better when it comes to richness of available libraries which is what node is missing. I think in real life node can handle more, I heard that issues with memory leaking are solved, probably uses less memory as well. I agree with conclusion that both are more then capable or handling async problems, I would use event machine on ruby end, this whole example seems flawed a bit. Anyhow, thanks for posting your results.