Question: (newbie here) if we assume that with 350 million keys for the 64GB ram server we have reached the peak of it's capacity, if we double the number of keys, do we have to get new hardware, like a second server or double the ram?

Or is it something simpler? Like storing the current keys on disk (somehow) and keep going with the server as it is.

To rephrase my question: keeping all in ram... is "all" like ALL ??? Would this solution work for the "real" twitter, with the billion twits, if 350 million keys were created every hour?

If you have more data than a single server can hold the best thing to do is hashing. For instance the Redis-rb client supports consistent hashing.

yes "ALL" in Ram :) this will work for twitter, facebook, everything: what they are doing is to use only the RAM even if they have MySQL, with tons of memcached around. With Redis of course the memcached layer goes away.

> If you have more data than a single server can hold the best thing to do is hashing. For instance the Redis-rb client supports consistent hashing.

Hopefully with a secret salt! It would be interesting if someone nasty managed to take down a node by e.g. predictively registering certain usernames.

Thanks a lot guys.

spoildtechie... that was an excellent suggestion. A quick browsing reveals that my question has probably been asked before... I had forgotten about stackoverflow... Thx.