For those talking about breakeven points and cheap cloud compute, you need to factor in the mental difference it makes running a test locally (which feels free) vs setting up a server and knowing you're paying per hour it's running. Even if the cost is low, I do different kinds of experiments knowing I'm not 'wasting money' every minute the GPU sits idle. Once something is working, then sure scaling up on cheap cloud compute makes sense. But it's really, really nice having local compute to get to that state.
Lots of people really underestimate the impact of that mental state and the activation energy it creates towards doing experiments - having some local compute is essential!
This. In the second article, the author touches on this a bit.
With a local setup, I often think, "Might as well run that weird xyz experiment over night" (instead of idling)
On a cloud setup, the opposite is often the case: "Do I really need that experiment or can I shut down the sever to save money?".
Makes a huge difference over longer periods.
For companies or if you just want to try a bit, then the cloud is a good option, but for (Ph.D.) researchers, etc., the frictionless local system is quite powerful.
I have the same attitude towards gym memberships - it really helps to know I can just go in for 30 minutes when I feel like it without worrying whether I’d be getting my money’s worth.
It depends on your local energy prices and your internet speed as well. It may actually be cheaper and faster to spin up a cloud instance. I bought an 80 core server to do some heavy lifting back in 2020. It doesn't even have a GPU, but it costs me around 4 euros per day to run. For that price I can keep a cloud GPU instance running. And even the boot up time isn’t any slower.
What cloud GPU instance are you talking about here? Most of the GPUs cost around 2 to 40 dollars an hour. I would love to know the provider who is offering one for 4 dollars a day...
That's a great point! I'd agree that just the extra emotional motivation from having your own thing is worth a ton. I get some distance down that way by having a large RAM no GPU box, so that things are slow but at least possible for random small one offs.
For those talking about breakeven points and cheap cloud compute, you need to factor in the mental difference it makes running a test locally (which feels free) vs setting up a server and knowing you're paying per hour it's running. Even if the cost is low, I do different kinds of experiments knowing I'm not 'wasting money' every minute the GPU sits idle. Once something is working, then sure scaling up on cheap cloud compute makes sense. But it's really, really nice having local compute to get to that state.