I think it's been said enough here but Varnish can certainly do almost everything that they are saying it can't do, including some other storage optimizations like storing the gzipped response and serving a non-gzipped when requested (as of Varnish 3.0).

You can use Vary for tons of caching optimizations via varnish, such as caching mobile web-pages vs non-mobile web pages or just a particular header. It's all about just flexing a little bit of VCL (which I'll admit sometimes can throw people off).

They had me until this part:

> This is an HTTP cache built directly in Passenger so that it can achieve much higher performance than external HTTP caches like Varnish.

And since they have no benchmarks to really back up these claims, I'm skeptical they did much research against Varnish to tune or set it up. I'd love to see the numbers on varnish vs their turbocache. Without numbers, I have to take a lot of it with a grain of salt.

Either way, seems like it could be an extra handy thing to have in your toolbox, as long as it fits your stack.

You're reading too much hostility in it. The article does not claim to be better than Varnish. Performance is achieved by not implementing many features. For example, Varnish is a full-blown programming language and has an infinite size. The Passenger turbocache has almost no configuration options, does not support any sort of custom programming and only supports 8 entries at most. The fact that the max size is so small allows it to be implemented in very compact data structures, but its usefulness is also severely limited. It's merely designed with a different set of tradeoffs than Varnish.

It isn't that difficult to make a web server that's faster all the production servers out there. For example H2O is faster than Nginx... because it has infinitely less features. Ditto for Passenger's turbocache.

The rest of the article describes some ideas, some of which cannot be implemented using pure Varnish and require cooperation from the app. In my opinion you're missing out on a lot of interesting ideas if you stop reading only because you think Varnish is being slammed.

I think the complaint is just that the approach is in no way as novel as you make it seem.

It can and has been implemented with Varnish and a few lines of VCL (with support of the app as well, of course).

What is "pure varnish" anyways? Proper varnish use always needs a tuned VCL, that's the whole core of the product.

Also, if I might say, I find your edits very handwavey. Please prove why that is impossible in VCL.

Fair enough. But even claiming that it's truly novel is not the point of the article. The point of the article is to research HTTP caching and to converse with the community about possibilities. It is not to present a new product.

In hindsight I see that the title may have been badly chosen. Before publication, none of the test readers have made any fuss about this.

> Please prove why that is impossible in VCL.

Where do you get impression that I claimed it's impossible in the VCL? It's pretty clear that the single page app approach can be replaced by edge side include, while the cookie parsing can be done with the cookie vmod.

The article claims that despite being the cache being able to do these things, application support is still required. The app has to be modified to output certain cookies in a certain format, e.g. the vary-on-user-permission-level thing. It's the combination that is important.

Have these things been done before? I'm sure they have. Most ideas in computer science are 30+ years old, and it's rare to truly find something new. But again, the point of the article is research. What we're after is not to present a new thing, but to present an idea, and if it's a new thing then we want people to help us to test; if it's not a new thing then we want to hear experiences.

Fair enough! I can definitely see how I might have taken it too hard (or it came off in my writing of the comment)

I was just more or less intrigued by the actual claim and would have loved to just see some numbers on "higher performance than varnish" and the scenarios where it could reach those performance pieces. It just piqued my interest for future architecture considerations.

Thanks for enlightening me more on turbocache. It's on my list of things to go through tonight.

Edit: I did want to point out my bad choice of words. "They had me until here.", did not mean I didn't read the article in it's entirety. Just that I continued with a bit more skepticism.

so @ tumblr, we're using varnish for a full page cache (we use it for parts of the API as well for response caching), and invalidate when a blog updates (or your page can just TTL out).

I definitely agree that I wouldn't use Redis (or memcache for that matter) for storing entire pages and should be used for more of an object cache. Even then, we use memcache for "simple" data structures and when we need more complex data structures, will use Redis.

Redis is great if you need some kind of persistence as well (and it's fairly tunable), where as memcache and varnish, are completely in memory (varnish 4.0 I believe is introducing persistence). So you kick the process, and that's all she wrote for your cache until it gets warm again. (Which has its own challenges).

Varnish also gives you a language called VCL to play around with to maximize your caching strategies and optimize the way varnish should be storing things. It's got an easy API to purge content, when you need to purge it and it should support compression for your pages out of the box without too much tuning.

If you're having issues just speeding up static content, give varnish a whirl. Spend some time with it, and you won't be disappointed.

I believe you can also look into using nginx as a caching alternative to cache responses, but I don't have too much experience with that. I've heard it used with some success though.

varnish has supported disk based caching for quite a long time, it's just that all the instances at tumblr are configured to only use memory.

Where can I report bad data for your service? Has all the wrong info for me, excluding my name and picture. I'm guessing it's got to be from LinkedIn?

Hey codingjester. Thanks for pointing that out! You can always hit reply if you see a problem with a report (we read every one).

Cool. I'll let you guys know. Thanks for the response.

For the Tumblr API, the link to the client is for our v1 API which has been deprecated in favor of our v2 API.

We have an official python client that I work on when I get time here: https://github.com/tumblr/pytumblr, Though I know there are tons of other awesome wrappers for v2 out there.

Shit. We totally didn't even think about mobile our bad. Once we get some sleep and probably get back from our day jobs, we'll see what we can do about it.

Not a huge problem, just polishing.

Thanks. We're updating it right now. It'll be up in a few minutes.

Maybe you can add gray text on input field saying 'enter your email' or something like that, that disappears when input field is clicked?

We're on it. Thanks for the help. We're putting it in right now and should be up in a few.