Sounds like great stuff! I think we shouldn't limit ourselves to source code directory structure so much and this is a step in the right direction.
One problem I foresee with this approach is that it could make debugging harder, if you have to look at the "concatenated" javascript file in production to make sense of what code's included and not, it's harder to know what comes from where than if you just included the original source code files.
One solution would be to have a way to switch between "concatenated" and "normal" modes easily (maybe it's already possible, I don't know), so that normal development would be unimpeded. But you'd have to resort to the big concatenated file for debugging in production.
Another solution would be to somehow preserve the information about what comes from where, so that via a browser plugin you could browse the concatenated file as if it were the original modular files.
Lately I'm getting really interested in ways to preserve information about "what comes from where" so that you could point to something in a generated output and ask the system to explain the sum of all inputs and considerations (decisions, transformations) that resulted in that "atom" being generated in the output through the various "semantic levels". You usually generate a low-level representation from a higher-level representation and it can be useful to proceed carefully from one transformation to the next so that you preserve meaningful semantic information about intermediate stages.
Cool, django-compress looks similar to the sprockets-rails plugin. A few differences:
django-compress makes you define your source files and concatenation order in an array in a configuration file. Sprockets lets you define your dependencies with annotations in the source files themselves, meaning plugins can have their own dependencies and integrate themselves with your applicaton's build system.
Sprockets doesn't do minification yet.
Sprockets doesn't do CSS concatenation, either. But it does have an asset provisioning feature, where plugins can specify a collection of files (e.g. css, images) that should be copied into the document root. (Rails itself provides basic CSS and JS concatenation.)
It's nice to see other similar but independent implementations of the same idea.
I dislike the comment "require" directives way of specifying dependencies. They should be real JavaScript function calls, which could either be analyzed ahead of time and stripped, like Sprockets, or used with a JavaScript loader system at runtime. This would remove the CGI/Ruby dependency and make the whole system more reusable (for example, server side, where concatenation isn't necessary/desired).
Can you elaborate? I assumed the problem was "JavaScript dependency management and concatenation".
What I described is essentially what we do in Objective-J. During development it just resolves the dependencies at runtime, but when you deploy you can flatten them into one big file.
I love how the options are Rails Plugin or CGI. If they write their marketing sites in PHP, why didn't they go with PHP instead of the CGI route? Or, is it the case that the CGI was just a "hey, you don't need to use Rails to take advantage of it" ploy?
EDIT: after reading the CGI source, they use some mod_rewrite trickery to avoid running the CGI if theres a cached file.
I originally wrote Sprockets 1) as a replacement for Prototype's build system and 2) as a library that could be used within Rails. Just before releasing 1.0, I added a simple CGI wrapper around the library so we could use it in our marketing sites and so people could use it relatively easily without Rails.
I like the idea of distributing a PHP wrapper. I'll try to add one for the next release.
And there's no "ploy." Interesting theory, though.
JavaScript resources are never downloaded in parallel. They are downloaded sequentially because they are interpreted while they download, and (existing) browsers always run JavaScript in a single thread.
This will have very little effect on caching. You will still need to configure your web server for optimal caching, by setting the Cache-control and Expires headers, for example.
Even so, this still has two main benefits: The JavaScript is minified, which will reduce the amount of raw data sent to remote users. Plus, by concatenating all the files together, you avoid the need for multiple separate HTTP requests.
I had previously been told that a JavaScript download basically halts the entire processing of the remainder of the page -- including any downloads that might occur there (CSS, images, and more JS) -- because the browser can't start parsing the rest of the page until it has interpreted the JavaScript. That's why it's best to move your JavaScript down to the bottom of the page.
Is this just the way older browsers used to behave? Are there still any restrictions like this that I should be aware of?
My rule of thumb with javascript is to put everything that is critical to page functionality right at the top and everything else as far to the bottom as I can (just before </body>), that way all your buttons/sliders and other active components will work as soon as the page is rendered and any other javascript will never hurt performance.
It seems that concatenating scripts could hurt performance in browsers that can do parallel script downloading, since it can no longer take advantage of the parallelism.
One problem I foresee with this approach is that it could make debugging harder, if you have to look at the "concatenated" javascript file in production to make sense of what code's included and not, it's harder to know what comes from where than if you just included the original source code files.
One solution would be to have a way to switch between "concatenated" and "normal" modes easily (maybe it's already possible, I don't know), so that normal development would be unimpeded. But you'd have to resort to the big concatenated file for debugging in production.
Another solution would be to somehow preserve the information about what comes from where, so that via a browser plugin you could browse the concatenated file as if it were the original modular files.
Lately I'm getting really interested in ways to preserve information about "what comes from where" so that you could point to something in a generated output and ask the system to explain the sum of all inputs and considerations (decisions, transformations) that resulted in that "atom" being generated in the output through the various "semantic levels". You usually generate a low-level representation from a higher-level representation and it can be useful to proceed carefully from one transformation to the next so that you preserve meaningful semantic information about intermediate stages.