Lots of people are focusing on this being done on a particularly powerful workstation, but the computer described seems to have power at a similar order of magnitude to the many servers which would be clustered together in a more traditional large ML computation. Either those industrial research departments could massively cut costs/increase output by just “magically keeping things in ram,” or these researchers have actually found a way to reduce the computational power that is necessary.
I find the efforts of modern academics to do ML research on relatively underpowered hardware by being more clever about it to be reminiscent of soviet researchers who, lacking anything like the access to computation of their American counterparts, were forced to be much more thorough and clever in their analysis of problems in the hope of making them tractable.
It's a good analogy. In particular, GPUs completely broke the trajectory of benefitting from carefully-managed compute granularity. One could say that CS/EE is now back-filling harder problems that didn't demand to be solved over the last decade because of GPUs.
Optimizing for cache management and branch prediction is very difficult. And most programming is done a level of abstraction that isn't amenable to staying portable (i.e. staying nimble) after optimization.
Plug: Staying algorithmically nimble after optimization is a problem we've "solved for" at our startup (monument.ai)
If anything it seems to me that doing the most work under constraint of resources is precisely what intelligence is about. I've always wondered why the consumption of compute resources is itself not treated as a significant part of the 'reward' in ML tasks.
At least if you're taking inspiration from biological systems, it clearly is part of the equation, a really important one even.
I find the efforts of modern academics to do ML research on relatively underpowered hardware by being more clever about it to be reminiscent of soviet researchers who, lacking anything like the access to computation of their American counterparts, were forced to be much more thorough and clever in their analysis of problems in the hope of making them tractable.