The point of this article is that Redis cannot be the "one true datastore". By "one true datastore", he means that Redis is the only data store you need in the stack, because it has an append mode that persists data in memory to disk.
The author's mistake is that he is setting up a strawman argument. I've yet to hear Redis claiming to be a end all be all solution for data storage. In fact, it would make sense that most production environments use Redis as the non-persistent cache layer backed by an ACID compliant datastore for the time being.
The author's second point is that Redis fails "at scale in production environments". I'm curious to see some examples here of production Redis failing and I'd like to know how those companies were using Redis. Either way, the scalability debate should wait until 2.2 is released, because 2.2 is supposed to address primarily multi-server Redis instances.
I think the main problem is just believing that there can be "one true datastore". Different technologies excel at different things. You pick what you want your app to do, and then take a look at the tools given to you and pick the best one for the job.
Redis happens to be a great choice for many things, but ACID compliant storage is not really one of them at the moment, although I suppose you could set up 2 Redis instances, one for fast in-memory usage and one that syncs to disk on every command for now.
From my experience, Redis is great for the following:
- Session storage
- basic LRU cache
- Publish / subscribe
- key value store where you want to manipulate data in the form of common data structures
- simple / medium complex queries
Redis is not good for:
- ACID compliant data storage AND fast in-memory access at the same time (the holy grail)
- Complex sql-like queries
Right now, if you try to make Redis persistent in the event of a crash (fsync constantly), you will get a performance hit because the writes to disk will be too frequent.
Antirez is trying to solve the problem of persistence while still keeping fast in-memory access. The author does not feel like Antirez will succeed here, and became "angry", which I find is a bit unreasonable.
I personally use Redis in production for what it's currently good for. If the persistence with speed issue is solved, all the better. It's not an all or nothing type of situation. The worst thing the author could do in this situation is to pick SQLite as your one and only data store for the entire stack. There's nothing wrong with having multiple types of datastores. In fact it is the norm and trying to fight against it at this time is kind of pointless.
I couldn't agree more. One situation I'm using redis for is loads of writes (fast ones) from a rails app.
Because this data adds up and I later need to query / report on it, I'm taking it out of redis after it reaches a certain age (every night) and bulk inserting (archiving) it into postgresql with a bit of aggregation. This gives me the advantage of super fast writes using redis, but also super fast SQL queries for reports that don't have to be real time.
This is probably more hassle than you want to deal with for most situations, but when you do need those fast writes, and you're not super concerned if you lose some data if the server craps out, it seems to be a nice fit.
Funny - I use amazon simpleDB/RDS in the exact same way - the index on RDS makes insertion really slow, so I use simpleDB as a cache and load into RDS nightly.
the connection time makes it relatively slow (on the order of a few milliseconds), but you can write to multiple keys at once, and/or multithread your client.
You wouldn't. Redis doesn't support implementing an LRU cache. Redis does function well as a cache, and is far better than memcached for any data you don't want randomly disappearing, but it just doesn't do an LRU cache.
Redis will only evict data when it's explicitly removed or expired, and it will store as much data as you put into it.
memcached will evict the least-recently used data based upon memory pressure, and will only use as much memory as you configure it to.
They overlap in functionality, but I find they work better in complement to each other than having one replace the other. Redis for data you want to persist, possibly with a timeout -- user sessions, for example. memcached for data you want to cache, as long you have the free memory for it.
You can configure a redis DB using the maxmemory flag instead of setting expiries, so yes it can be used as an LRU cache.
Probably not as fast as memcached, but one situation where it would be appropriate is if you're already using redis in your stack for something else (fast writes). Rather than adding another piece of complexity to your stack you can instead get double use out of redis as a cache as well.
Why wouldn't you? As I understand it, Redis is a superset of memcached (since you can turn disk writes off) and comparable in speed. I, at least, use it as a memcached replacement for storing sessions with the added benefit of not having them all expire if I need to restart the server at some point.
Basically, I exactly agree with the GP's use cases. ACID for things I want to keep, redis for small things that need to be fast (sessions, stats counters, etc).
Agreed on that. Plus, i noticed a mistake made by a lot of people, Redis is NOT memcached. That's just as easy as that. You can use Redis the same way as you would use memcached, but that would be a waste.
The power of Redis resides in its data structures and the usage you make of it. You can make some pretty powerful stuff, easily and with performance.
Nested Comment system (https://gist.github.com/803251), autocomplete (http://pastebin.com/E5h1TLWz), all of that with high performance over large dataset, and much more, stuff that is not doable with Memcached. Also, the role of Redis is not to replace SQL databases, i personally view it, and use it, as a complement to it, and it works just perfectly. (I am fully aware that the examples given are possible with SQL solution, but experience in production/development as shown me that Redis is easier and faster when it comes to not too complex operations like these ones)
Also, complex queries should be handled by performing them periodically on your backend stroage offline, storing the results in something like redis and then simply read from the redis DB online.
I think simple redis queries have the perfect level of data logic you would need just to be able to retrieve the corect/relevant results from the complicated queries you produce.
There are so many people who have written about methods on how to do something like this, eg. facebook, disqus, etc.
Agree, I'm using:
PostgreSQL for normal SQL
Memcached for non-persistent fast changing cache
Redis for session storage, slowly changing persistent cache
The author's mistake is that he is setting up a strawman argument. I've yet to hear Redis claiming to be a end all be all solution for data storage. In fact, it would make sense that most production environments use Redis as the non-persistent cache layer backed by an ACID compliant datastore for the time being.
The author's second point is that Redis fails "at scale in production environments". I'm curious to see some examples here of production Redis failing and I'd like to know how those companies were using Redis. Either way, the scalability debate should wait until 2.2 is released, because 2.2 is supposed to address primarily multi-server Redis instances.
I think the main problem is just believing that there can be "one true datastore". Different technologies excel at different things. You pick what you want your app to do, and then take a look at the tools given to you and pick the best one for the job.
Redis happens to be a great choice for many things, but ACID compliant storage is not really one of them at the moment, although I suppose you could set up 2 Redis instances, one for fast in-memory usage and one that syncs to disk on every command for now.
From my experience, Redis is great for the following:
- Session storage
- basic LRU cache
- Publish / subscribe
- key value store where you want to manipulate data in the form of common data structures
- simple / medium complex queries
Redis is not good for:
- ACID compliant data storage AND fast in-memory access at the same time (the holy grail)
- Complex sql-like queries
Right now, if you try to make Redis persistent in the event of a crash (fsync constantly), you will get a performance hit because the writes to disk will be too frequent.
Antirez is trying to solve the problem of persistence while still keeping fast in-memory access. The author does not feel like Antirez will succeed here, and became "angry", which I find is a bit unreasonable.
I personally use Redis in production for what it's currently good for. If the persistence with speed issue is solved, all the better. It's not an all or nothing type of situation. The worst thing the author could do in this situation is to pick SQLite as your one and only data store for the entire stack. There's nothing wrong with having multiple types of datastores. In fact it is the norm and trying to fight against it at this time is kind of pointless.