Likely culprits are "performance analyzers" that grade a website and report an "F" (failing) grade for not using CDN-hosted common libraries.
This is a red herring: this idea that the user will already have a cached copy of CDN-hosted jQuery is bogus. Even for a common library like jQuery: the number of versions of jQuery that are in use is likely above 50, and the number of popular CDNs that host jQuery is surely above 10. So we are hoping that the user will have a cached copy of that exact jQuery version from that exact CDN.
This is somewhat similar to the situation we have with operating systems: we created shared libraries to save disk space and memory. These days using them is pretty much pointless and incurs a performance penalty, yet everybody still uses them.
For JavaScript, a much better approach is to a) make code Google Closure-compatible, b) compile everything using advanced mode into a single JavaScript file. That way you get an optimized subset of all the code that the site actually uses (this works wonders for ClojureScript apps). Most sites probably use less than 10% of jQuery, so why include all of it?
> These days using them is pretty much pointless and incurs a performance penalty, yet everybody still uses them.
Would you rather than when (e.g.) there is a security patch for OpenSSL, that you have to wait for all software using OpenSSL to deploy updates? Or would you rather that one update to OpenSSL (likely from your OS vendor) fixes all of the software depending on it?
Edit: People seem to be commenting to this through the lense of CDNs and JavaScript, but the sentence previous to the one I quoted was:
> This is somewhat similar to the situation we have with operating systems: we created shared libraries to save disk space and memory.
Which is not talking about CDNs and JavaScript, but shared libraries on your desktop. I'm not saying that all usage of shared libraries is valid. I'm just saying that to toss out the concept as entirely useless (and having no redeeming value) in a modern setting varies from the truth.
You get pretty close to full caching... often better than using your own copy. The reason it isn't a common practice is more about potential bugs caused by newer versions.
> This is a red herring: this idea that the user will already have a cached copy of [open-ssl] is bogus
He says this is because of many different versions in use
While this isn't true for a managed repository of software, it is still true for most software releases so the mismatch just might happen further down the line.
I did a little searching to expand on your numbers. What follows is not scientific.
According to https://www.datanyze.com/market-share/cdn/ CloudFront, Akamai, MaxCDN, CloudFlare, EdgeCast and CDNEtworks account for ~75% of CDN usage by the Alexa top 1M (with 28 others listed)
Data from http://trends.builtwith.com/javascript/jQuery
suggests that the 1.4.2, 1.7.1, 1.7.2, 1.8.3 versions of JQuery are cover 53% of the 23M the have version data for (with 23 others listed)
That puts a lower bound of 690 on the number CDN-Version pairs in the wild.
If we make the (totally unsupported!) assumption that the distribution of versions is the same across all CDNs than 20 of these CDN-Version account for ~25% of the versions.
This could suggest that there is a cache advantage towards using Jquery 1.4.2 (21%) served by Akamai(37.5%)
Seems like jacquesm should now have the data to test this and give us a actual answer.
>This is a red herring: this idea that the user will already have a cached copy of CDN-hosted jQuery is bogus. Even for a common library like jQuery: the number of versions of jQuery that are in use is likely above 50, and the number of popular CDNs that host jQuery is surely above 10. So we are hoping that the user will have a cached copy of that exact jQuery version from that exact CDN.
I wonder if it might be a good idea to have a hash attribute for external resources. For example, I might include jquery by including
I calculate the hash on my end. This ensures that if code.jquery.com/jquery-1.11.3.min.js is changed, the browser can know that the resource was tampered with or the developer made a mistake, and not load that resource. Also, if the browser sees a hash for a resource it has cached, it can load that cached resource, even if it is hosted at a different location. This seems better for both security and performance, but does put a slightly higher burden on the developers.
Can you please elaborate how using share libraries is "pretty much pointless and incurs a performance penalty"? That goes against my intuition of how they work.
Think it means that most machines are not memory or disk constrained these days, but there is extra processing to perform the dynamic linking. Sort of a cost-benefit argument it seems. Doesn't address the issue of security etc and the benefit of just having 1 instance of a library to update when maintaining a complete system however.
[I should not have brought shared libs into this, I now regret it, because it sidetracked the entire discussion, but...]
The extra processing is more significant than most people think. The library has to be compiled as relocatable code, which incurs a runtime performance penalty. You also lose a register, which especially on register-constrained architectures is really bad (it was a tragedy on iA32, it's less of an issue now).
What if we add kernel same-page merging to the mix? Might still be a little less efficient at run time than the optimal use of shared libraries. But shared libraries make packaging more complex, especially if one does it Debian-style, with each shared library in its own package, a separate -dev package, etc.
> What if we add kernel same-page merging to the mix? Might still be a little less efficient at run time than the optimal use of shared libraries.
Might be a whole lot less efficient than even sub-optimal use of shared libraries. A optimizing linker pulling together static libraries is going to make page-merging the executable almost impossible.
I remember reading about http2 that using a single javascript file is an anti-pattern since http2 has smarter management of requests and can deal with them in a more granular manner.
That's a big chunk of the Internet right there. IE 11 and Safari 9 both support it, so once their respective betas go public that's the rest of the client-side support. Nginx is supposed to support it by the end of the year; once that happens most sites will get it just by tweaking a config file:
Only for TLS or only in beta versions. It's still going to be awhile before it's worth it to sabotage older browser performance, even once sites update their servers.
And in the end, that'll just embolden sites to crust their pages with more analytics and trackers until the performance isn't any better.
If you broaden that out to http2 and its very similar predicessor spdy then the browser support graph looks even better, including the latest versions of Safari, Mobile Safari, and IE: http://caniuse.com/#feat=spdy
Actually no. If you advertise in your headers that you support SPDY/HTTP2, they'll use it even if they are not using encrypted http in the first request. Anyone who hasn't updated their servers to support it can't honestly claim they care a lot about performance.
> It's still going to be awhile before it's worth it to sabotage older browser performance, even once sites update their servers.
It is already worth it really, particularly when you factor in mobile where performance is a bigger.
> And in the end, that'll just embolden sites to crust their pages with more analytics and trackers until the performance isn't any better.
Well, there is a natural equilibrium that we tend to arrive at, but at least 1st party trackers are so lightweight with SPDY as to be irrelevant. If you have one 16K image somewhere on the page, the overhead of loading 100 trackers will seem negligible (JavaScript might be another matter though ;-).
How would the closure compiler figure out what bits and pieces of the library are triggered from the html portion of the site? (I can see how it can track the javascript bits but unless your site is entirely generated from js you'd have to start with the html)
Well, it's something we should look at, especially the library makers.
People who write ClojureScript regularly encounter this: we get this optimized and trimmed down app, and then something needs jQuery, so we have to pull all of it in. After you've used advanced compilation for a while, it feels downright dirty and wasteful to pull in entire blobs of code, not just the function trees you actually need.
You can also define a separate externs file with a list of symbols that shouldn't get mangled; this is useful if eg. you're using a third-party library that you don't want to modify.
Google Closure compiler looks at all of your JavaScript at once and throws out whatever isn't actually used. So, if you call your JavaScript from HTML somewhere else, you need to explicitly list the functions you use.
Check your browser cache sometime. You no doubt have most of the CDN-hosted jQuery's in your cache. You also probably have hundreds of copies of Closure compiled jQuery. ;-)
> Minifying your JS and CSS files is a very good practice as it's not only secure, but also is compact.
If you think you gain much in terms of compactness, you might not understand how the subsequent gzip compression works. ;-) There may well still be a gain, but it won't be significant.
As an example, on a previous project all of the javascript libraries + the app concatenated together at 3MB. Minification with uglify reduced the size to 1.5MB and using Gzip compression further reduced the transfer size to ~800K.
Have you compared that to the transfer size of gzip without a minification step? I haven't seen a real difference, myself, and I've been considering taking the minification step out.
This is a red herring: this idea that the user will already have a cached copy of CDN-hosted jQuery is bogus. Even for a common library like jQuery: the number of versions of jQuery that are in use is likely above 50, and the number of popular CDNs that host jQuery is surely above 10. So we are hoping that the user will have a cached copy of that exact jQuery version from that exact CDN.
This is somewhat similar to the situation we have with operating systems: we created shared libraries to save disk space and memory. These days using them is pretty much pointless and incurs a performance penalty, yet everybody still uses them.
For JavaScript, a much better approach is to a) make code Google Closure-compatible, b) compile everything using advanced mode into a single JavaScript file. That way you get an optimized subset of all the code that the site actually uses (this works wonders for ClojureScript apps). Most sites probably use less than 10% of jQuery, so why include all of it?