It should be noted that this post was written over one year ago and we are not using polling any longer, but comet solution.
We wanted to do realtime notifications and polling isn't a way to do this. We tried different options and in the end wrote our own comet solution using JBoss Netty. Working with JBoss Netty was really great, but we hit perfomance limits on lots of open connections (100.000+).
At that time node.js was released and I did some experiments with it and found it it had great perfomance and very low memory footprint. I also had an university course on the V8 VM (by the actual Google team :)) so I was pretty interested in V8 and node.js at that time. I experimented more with this and eventually we rewrote our comet solution to node.js
Fast forward today we are still running node.js and processing many millions notifications each day. I would recommend digging into node if you are interested in building a scaleable notification system. Our comet implementation for node.js is under 1 KLOC.
Our node.js solution uses around 10x less memory than the Netty solution. It also serves many more req. pr. sec on a lot of open connections. Of course, Netty might have improved since we used it and we might have used it wrongly. Generally my experience was good with Netty and buying a lot of memory is cheap, so Java developers should definitely check it out.
A quick note is that we have implemented a custom java.nio solution and it had similar memory usages like Netty. It was much harder to code, so I would not recommend using java.nio directly, especially since there is little to gain in terms of perfomance/scalability.
amix,
i am sure with the suggestions that Zed made and from your own experience of using that in production you would have made modifications to the code.
is there any way you can post the code that you last used in production?
What is the bottleneck here? I am surprised that the CPU utilisation is just 2%, which means, either the disk or the network is the bottleneck. Both these aren't discussed. And of course, one shouldn't forget the fact that if CPU is the bottleneck, we need "true" concurrency and there should be threads on every single core. :-)
Hitting a non-blocking C stack is much better than hitting a blocking Python stack - especially when you have tons of clients hammering with polling requests. So the optimization was to reduce the load on our Python servers by handling most of the polling requests in a C based server.
The mistakes are harmful. When you mess up your string handling in Perl or Python, you end up with double-encoded utf8, or something. When you mess up your string handling in C, other people on the Internet get a shell on your computer.
C is serious business.
(Incidentally, I've written event-driven programs with libev in both C and Perl. My load is usually so low that doing it all in Perl does not affect the number of requests I can handle, nor does it impact latency. The IO is the bottleneck; the article even says it only uses 2% CPU. Why risk insecurity when you don't know C and it doesn't even improve throughput?)
I'm interested to hear other peoples' thoughts on comet versus polling. I've done polling solutions before and am in the process of setting up a comet based solution.
In my opinion, having fewer requests to the server is better. But if someone can combat me on this, I'd really be interested in hearing another voice.
We wanted to do realtime notifications and polling isn't a way to do this. We tried different options and in the end wrote our own comet solution using JBoss Netty. Working with JBoss Netty was really great, but we hit perfomance limits on lots of open connections (100.000+).
At that time node.js was released and I did some experiments with it and found it it had great perfomance and very low memory footprint. I also had an university course on the V8 VM (by the actual Google team :)) so I was pretty interested in V8 and node.js at that time. I experimented more with this and eventually we rewrote our comet solution to node.js
Fast forward today we are still running node.js and processing many millions notifications each day. I would recommend digging into node if you are interested in building a scaleable notification system. Our comet implementation for node.js is under 1 KLOC.
Other posts you might be interested in:
* http://amix.dk/blog/post/19484#Comet-with-node-js-and-V8
* http://amix.dk/blog/post/19456#Plurk-Comet-Handling-of-100-0...
* http://amix.dk/blog/post/19489#Comet-long-polling-for-all-br...