I'm not sure why individuals feel the need to write an entire page of text regarding a relatively straight-forward bug fixes. "Improved [Rails|Ruby|...] performance by [X] ms by changing [N < 20] lines of code" posts are becoming a common occurrence.
The regular occurrence of such issues in a project implies an immature, poorly profiled code base, and are the kinds of things that other projects fix without fanfare.
Our most recent database-backed dynamic web application can serve >10000 requests/sec on our mid-range hardware at <2 ms/req, which we achieved by running the most basic of profiling early on and throughout the development process, and using software that does the same. I don't expect a gold star for this.
You've missed an important but subtle point. The article points out that the reason this was overlooked is not because Rails isn't profiled but because this behaviour is happening outside the standard rails profiling scope. That is a) why it's not a "relatively-straightforward bug fix", b) why it hadn't been caught already and c) why it is interesting. Actually I take that back, it is a relatively-straightforward bug fix. As with all the best bugs it's finding it that wasn't straightforward.
I think a similar thing happened about a year ago when people were profiling rails but not profiling memory usage. After they realised GC was impacting their performance they shifted their focus and improved things in that regard. The most recent "X line Y% speedup" article I read here took this even further and examined the impact of the size of stack vs. heap memory on GC times.
My particular favourite example of this is Steve Souders work showing how much your web app performance is affected by the end users browser and its wacky network and cache behaviour. He has claimed these factors can account for 80-90% of the end-users' waiting time. I know rails core have been working to fix these issues as they did a presentation about.
That sounds to me like a bunch of useful lessons to be learned from. Certainly more info than just "profiling good", more like non-holistic profiling can be misleading. Your response certainly doesn't cover this as you talk about "basic profiling", so what is outside the profiling you do? How would you know if it is affecting performance and to what degree?
You've missed an important but subtle point. The article points out that the reason this was overlooked is not because Rails isn't profiled but because this behaviour is happening outside the standard rails profiling scope. That is a) why it's not a "relatively-straightforward bug fix", b) why it hadn't been caught already and c) why it is interesting. Actually I take that back, it is a relatively-straightforward bug fix. As with all the best bugs it's finding it that wasn't straightforward.
Properly profiling the full execution path should be an entirely obvious step to take.
The short and to-the-point version of the post went to the Rails-core mailing list. On my blog, I prefer to tell stories.
Bragging about how fast you can serve a request is really meaningless when there's no way to know what's involved in that request. On my personal side projects (also using Rails), I serve requests in a couple milliseconds too. But, there's a vast, vast difference in what those sites do, vs what's involved in running a large-scale e-commerce operation.
The short and to-the-point version of the post went to the Rails-core mailing list. On my blog, I prefer to tell stories.
There are quite a few stories of this nature, which could simply be summarized as individuals finding incredibly simple inefficiencies and then fixing them.
IME, many many developers have no idea how to measure and profile their code. The reason for the stories is to help educate people with less experience in what tools to use, and how to use them.
I'll admit that I don't know enough about this. Got any pointers or links for the intermediate Rails hacker? (I'll check out ab, as mentioned in your post.)
I like this type of blog post. I'd much rather read stories about people solving real problems with real code than blowhards pontificating about their favorite design patterns.
Whether Rails is immature doesn't make it less interesting to see how people track down its problems and fix them.
Having lived in Ruby code for a bit, I think it's fair to say that you're entirely right. It's immature, it's poorly profiled, it has memory leaks, and the scoping rules are a nightmare.
But it sounds like you were doing profiling and optimization early and often, which might have been premature, and thus might have been a waste of time.
Who's to say? Ruby isn't for everyone. Perhaps your ideal web stack is what you're using. (What're you using, btw?) I've been itching to use Haskell for a webapp.
"Perhaps your ideal web stack is what you're using. (What're you using, btw?)"
He has stated in previous comments that he's using scala.
Interestingly, merb on jruby has reportedly reached 14000 rps on similar hardware to what he's refering to (8 cores, since he previously claimed to get 5000 rps on a 4 core machine).
The regular occurrence of such issues in a project implies an immature, poorly profiled code base, and are the kinds of things that other projects fix without fanfare.
Our most recent database-backed dynamic web application can serve >10000 requests/sec on our mid-range hardware at <2 ms/req, which we achieved by running the most basic of profiling early on and throughout the development process, and using software that does the same. I don't expect a gold star for this.