With defer_javascript on, browser-reported page load times tell less of the picture. The problem is that what browsers report back to your analytics service is the amount of time that passes before the onload event while defer_javascript [1] postpones javascript execution until the onload event. This means that with defer_javascript off you were counting javascript execution time but when you turned it on you stopped counting it.
We're trying to optimize something like "time until the page is visually complete and usable", and there's not currently a good metric for that. Speed index [2] does visual completeness well, but I don't know of any algorithmic way to measure time-until-usable.
(I work on the PageSpeed team at Google, mostly on ngx_pagespeed.)
Checking with some people here, it's actually more complicated than this: what I described is true for IE9 and below, but for Firefox, Chrome, and IE10 measured page load time will include execution of these deferred scripts.
Google Analytics site speed data is only based on 1%[1] of your traffic while New Relic's Real User Monitoring aims to track all of your traffic. I'm sure that can lead to differences in accuracy.
> Not to be confused with mod_pagespeed, which is an Apache module
PageSpeed Service proxies your site, but aside from that the optimizations it makes are very similar to mod_pagespeed (and ngx_pagespeed). Which makes sense: they're closely related Google projects.
(I work on all three, but mostly ngx_pagespeed these days.)
I'm not sure the "cloudflare has been getting worse and worse" that links to a single page written by you referencing a single twitter post. Really helps the article any.
Obviously I have little to no idea how many sites I visit daily are using CloudFlare, however:
* Over the last two weeks, the number of CloudFlare 'down' pages I have seen has grown quite rapidly on sites that were never unavailable in the first place. (And, typically, the 'live' link ends up at the correct, working, page)
* imgur load time has gotten exponentially worse since the addition of CloudFlare. (Whether or not performance would have degraded the same without it is clearly debatable.)
The first is that you have to be careful with the 'live' page stuff on Cloudflare and blaming it directly on Cloudflare. I can't know the exact circumstances that lead you to see the error, but in our experience the couple times that we've had a problem with this, it's been an actual issue on our side that wasn't presenting itself at great frequency, but enough to trigger a live-site error once in a while.
As for imgur, I'd have to look around their site, but I'd say perhaps they didn't do their homework completely? I mentioned it on a previous post but you have to be extremely careful with setting headers for objects or every request will be proxying for an image from Cloudflare, back to the internet, then to your servers, then back through the internet, through Cloudflare and then finally to the user.
Instead, you want to make sure that Cloudflare is caching mostly everything EXCEPT for the page body if at all possible. A developer accidentally turned off proper headers a while back and well, we noticed it in page load speed issues pretty quickly.
Edit: Also, you probably want to see if you can cache the page body as well ;).
you have to be extremely careful with setting headers
for objects or every request will be proxying for an
image from Cloudflare, back to the internet, then to
your servers, then back through the internet, through
Cloudflare and then finally to the user.
I'm pretty sure PageSpeed Service will do this too if you set headers to prohibit caching (which is depressingly common).
I imagine that would be correct, yeah. If you don't specify to cache something, it shouldn't be cached. If a proxy out there still caches it, I'd consider it broken.
From memory, if something has no caching headers PageSpeed Service will treat it as
Cache-Control: public, max-age=600
Combined with serving html as uncacheable, this is a pragmatic and functional way of dealing with the many sites that haven't put much (any) thought into caching.
Ah, interesting. Thanks for noting this. Something I'll keep in mind if we ever look at the PageSpeed service. I'm actually not certain if Cloudflare is doing something similar for pages with no headers set.
It's good to hear some insight into this, as it's really been grating on me lately, so thanks!
I'm somewhat curious about what kind of problem you could have that would allow the 'live' site to work, but break through CloudFlare. That seems entirely counter to the functionality... do you happen to remember or are you able to comment on what was happening?
Sure. There are at least a couple ways I've seen that it can be triggered (and if one of the Cloudflare guys on HN is around they might have more or correct me if I'm wrong).
The first seems to be that if a page is experiencing slow loading times on the upstream locations (and pretty much everyone using Cloudflare is likely just an 'upstream' entry in nginx), you can trigger the error. If the upstream server returns a 500 error I have seen certain circumstances where that will return a live page error... or, if the web-server on the remote end terminates the connection for whatever reason that will also cause it to happen.
For us the last time we had a problem it turned out to be pretty complicated. There was a bug (which we reported and was fixed in the last release) for the php NewRelic module. Whenever there was what the module considered a 'slow' mysql query, it would cause a segmentation fault. This particular day there was randomly a few 'slow' queries, thus when the segmentation fault occurred the php thread would die killing off the connection and bringing up the 'page is down' error from Cloudflare. That was a pain to diagnose. Strace for the win.
No offense, as I know little about your website, but you're one site out of thousands/tens of thousands on Cloudflare at this point. Your results without substantial research and supporting evidence of this being a trending problem with other sites is not enough to say (per your article) 'As I mentioned earlier, avoid Cloudflare at all costs'.
First, have you contacted Cloudflare support regarding your purported page slowdowns? If not, they are actually very good about both fixing things and getting back to you, even on the piss-ant level plans.
Second, what optimization features exactly do you have enabled? Do you set proper cache headers? Those are especially important to Cloudflare and without them being properly set (especially on JS and CSS objects) I imagine your page speeds will be pure crap.
That isn't to say that Cloudflare is without issue. We've had them, they exist. But if you're going to present an issue and blame it on the service, do your homework first and present full evidence, otherwise don't spread it as it's just an unproven lie.
Yes, I have contacted Coudflare about slowdown issues. I tried to use Cloudflare across a few sites with many weeks of testing. I agree that they are extremely helpful and responsive. One specific support ticket was about the time it took for ajax requests to complete. Cloudflare was adding up to one FULL second of latency. I also mentioned response times reported by GoogleBot. They had no real answers. Though they did acknowledge some Ajax slowdown that was supposedly fixed.
Some days are better than others, but overall Cloudflare did not speed anything up. Perhaps free users don't get as good of a performance boost?
I had similar features enabled with Cloudflare as I do with Google PageSpeed. The sentence right before the one you quoted says: "The response times to the far left are a result of Cloudflare’s Full Caching features with Rocket Loader enabled.".
Essentially that graph shows what happens when all of Cloudflare's "CDN + Full Optimizations" features are enabled compared to most of Google's PageSpeed Service. It's the closest comparison that I can make.
Hard for me to say what happens for 'free' accounts. We only use pro or higher (A good number higher than pro). I would say not counting the additional performance features you get at the pro level, if they're purposefully slowing things down for free accounts that would be pretty crummy.
That still doesn't answer what you're doing with your cache headers. I took a basic look at your fantasysp site and I wonder if there isn't some changes you could make on the cache-control settings. You seem to be setting no-cache quite a bit.
You may want to avoid setting Expires to greater than 1 year, RFC 2616 (HTTP), section 14.21:
"To mark a response as 'never expires,' an origin server sends an Expires date approximately one year from the time the response is sent. HTTP/1.1 servers SHOULD NOT send Expires dates more than one year in the future."
If an agent is precisely following the RFC, anything set to more than 1 year in the future is an invalid date and:
"HTTP/1.1 clients and caches MUST treat other invalid date formats, especially including the value '0', as in the past (i.e., 'already expired')."
I just realized actually one very likely reason you would see a significant increase like this. I imagine that the GoogleBot and Pagespeed servers are on the same network, or at least regionally close enough to make a big difference.
Cloudflare is not going to be on the same network as Google for any of their servers as they host their own gear around the world. If you want to see the real speed decrease from switching it should be a set of tests between Cloudflare and Pagespeed from outside both of their respective networks (and a couple locations if possible). That would give you closer to real numbers... or the google services are on totally different networks and what you saw was real-world, which I couldn't answer.
Sorry, not about to reveal realistic traffic counts (also this is getting off-topic)... but I will say that we generally push somewhere around 120-160Mbit/s.
It's on-topic because you're presenting yourself as an authority on Cloudflare's reliability then dropping small numbers to back it up. 150 Mbps isn't really interesting as a data point on speed/reliability either. What you've described so far fits in a single rack several times over.
It would be interesting to know what the before/after load time histograms look like. The averages that New Relic give can hide a lot of the details of what's actually happening.
It's a great article, thanks a lot for sharing this.
The most amazing part of all of it is that Google offers it for free while many other companies charge a lot of money for the same (or even worse) functionality.
PageSpeed Service is currently offered free of charge to a
limited set of webmasters during this trial period. Pricing
will be competitive and details will be made available later.
You can get an idea from what they are charging for PageSpeed on App Engine. It may be competitive but it's not cheap. Perhaps they've learned that price increases aren't taken well and have priced it high with the possibility to reduce it when it's widely available.
Sorry if it's a bit scatter-brained and unorganized. I threw it together late last night and am posting to help show others what to expect.
"no understanding from author about how the services actually work". I gave my interpretation of the data I was presented with. I am no expert in deferred javascript.
If you'd like you can ignore the entire article's text and just look at the pretty graphs.
We're trying to optimize something like "time until the page is visually complete and usable", and there's not currently a good metric for that. Speed index [2] does visual completeness well, but I don't know of any algorithmic way to measure time-until-usable.
(I work on the PageSpeed team at Google, mostly on ngx_pagespeed.)
[1] https://developers.google.com/speed/docs/mod_pagespeed/filte...
[2] https://sites.google.com/a/webpagetest.org/docs/using-webpag...