I've no doubt memory is slow, taking up to 100-300 cycles in NUMA systems. But a single threaded server accessing local memory won't may those costs as much. Are you saying that a few random memory accesses are slower than sending and receiving a packet on two machines?
Redis has a great position as a persistent, shareable, data structure server, but replacing in memory hashtables where they work doesn't seem like one of those cases.
Sure... on a 10 Gbps link, you can transfer up to 15 million packets per second (assuming a tiny payload... which is about all you need for a hash lookup). That's about the same order of magnitude as the latency of a single memory lookup that misses the cache. In the right environment (not TCP on a stock Linux kernel!) each packet carries very little CPU overhead.
Of course, you can pipeline memory accesses to some degree, but not as easily as you can aggregate network requests into fewer packets.
I'm certainly not saying the network overhead is free -- it's not! I'm just saying it needn't "eclipse" the hash table lookup itself (as the GP suggested). They're on the same order of magnitude.
Alright but in the case of Redis, this low latency networking like RDMA or whatnot isn't available right? So, today, off-the-shelf, installing Redis on a remote server, you're probably gonna be off by at least an order of magnitude, no?
Also, on the target server, you still have at least one cache miss to retrieve the item from the hashtable (and potentially more for large hashtables).
It would be nice if there was a commonly supported API for doing this kind of stuff that doesn't require exotic hardware and came with a usable abstraction something like TCP. I know off-the-shelf Intel NICs have "Direct NIC Access" which can easily do wireline work from userspace, but afaik that's only for Intel NICs, and the API isn't as smooth as most socket users are used to. We need something like SuperSockets, but written to target NIC hardware directly, I guess.
Redis has a great position as a persistent, shareable, data structure server, but replacing in memory hashtables where they work doesn't seem like one of those cases.