Oh I think I get it, instead of waiting for a user to click on a page and then call the server to render that page, the flat file method renders all the pages at once w/o worrying about which pages the user will click on... Correct?
I say this in "how far have we come" "we've come full circle" and not in any way as disparaging.
As someone who remembers Perl CGI's ability to serve dynamic content as being revolutionary and awesome. It amazes me to hear someone who's only experience is dynamic content and needs http file serving explained to them.
Well, I learnt Java's oop first and then C structs. So to me a C struct is just like a Java object, only with no methods in it. My poor TA couldn't get over it.
I have the same feeling. The new generation takes dynamic content being generated on the fly for granted, even though in many cases it's not necessary at all.
It really is as simple as it sounds: files. It's just HTML and CSS and JavaScript and what not. There's no server running, except to (re)generate that HTML when it's out of date.
It takes a different way of thinking because you have to design your html + js to work for everyone that accesses it.
It is still possible to personalize the content a user sees because you can still authenticate them and use ajax GET/POST requests, but you have to do all that with javascript.
You still need a server, but instead of your server generating the full html for each request, it only needs to generate a json response for some percentage of total page views. This is just lighter weight, and the architectural approach we are heading to with backbone.js et al...
Basically you offload as much of the computational load to the client side AND the batch rendering process as possible.
One reason this was important for my purposes with streamjoy is that even though 92000 movies isn't Big Data, it still takes a long time when you have to run 10 kinds of algorithms on 92000 items AND deal with the restricted API limits of a big API like Amazon (about one request per 1.8 seconds unless you get special permission or are doing a lot of revenue)
So for streamJoy I had to do a lot of computation and network queries, and I had to respect the 24 hour period that Amazon wants for data freshness.
So the batch jobs I had to run had to be as fast as possible, and that means the fastest CPU + SSDs + ample RAM.
A dedicated i7 server with 32gb of ram and SSDs is $180 at the cheapest data center I could find. Thats way too much for a portfolio piece!
So I offloaded all the batch computation to my own local workstation and then just push the rendered results to the cloud.
Thats one of the big benefits of the flat file approach.
