Have you worked at an enterprise? Everything is about money.
Don't forget the scale at which these operate. $1000 extra in a GPU adds up quickly when you buy a thousand of them.
Perhaps in big tech it's different but in our huge multinational anything IT is seen as a cost center only and has to be on the cheap. Which a real focus on short term savings because who knows if this year's SVP will still be in that seat next year.
Nobody ever told me "let's take the most expensive option because it's the best in the long run" :(
That's kinda what top parent comment is talking about though. Money doesn't matter when you only have one choice. Enterprise can ONLY buy Nvidia cards so money really doesn't matter in this case. Technical execution is more important than anything else, including money. If you can't afford or make profit to buy cards to support what you need (more VRAM in this case), the project is dead in the water to begin with.
You should never compare an additional cost to the overall c ost in a production setting. You should compare it to the profit margin. If you are running a 10% margin, then a 2% cost increase is -20% profit, a quite substantial amount.
If employees are more efficient, I need fewer of them. People cost a lot of money relative to capital expenses like nice laptops, good chairs, or individual GPUs.
This kind of spending almost always pays off very quickly.
Yes, I also believe that. The only thing I argue against is the 'negligeable' marginal cost statement fed to devs by dev gurus because it is a fallacy.
I'm all for investing in quality work experience. Decent equipment being the most obvious. Not only for the potential higher productivity, but also for its definite hr/retention effects. Investing in tangible production improvements and comfort is money spent 10x more effective than the hr semi-mandatory fun/social events that have metastasised throughout tech.
Where asking wasn't an outcome, in contrast to some other colleague that just went "I guess we will just deliver below potential then" I used to buy my own stuff because I respected my time, but it definitely didn't want to keep me working there.
Now this does not mean you have to shell out for every whim. If you want a 10.000€ PC setup instead of a 4.000€ one, I will need a better justification than 'custom loop and RGB'.
Let me be very clear: Because I have a fancy GPU in my computer, I was able to prototype algorithms in my spare time (e.g. nights and weekends), learn a lot more about the state of machine learning, and use this in production.
I can think of one specific use -- processing a data set -- which allowed me to do something automatically which would have used a few hours manual time. My hourly salary times hours saved on that one use already paid for the GPU.
I use Stable Diffusion many times to make images for presentations before it was mainstream (yes, there were a few months before everyone was using DALLE and SD). That had a noticeable impact which went well beyond the cost of the GPU.
And so on.
Yes, I could have done this at lower cost on cloud-based services with only a little bit more overhead, but:
1) I wouldn't have put in the extra work
2) If I did, I wouldn't have been able to justify the expense when things were at the half-baked stage
Things like Hugging Face are at a level where this DOES increase productivity by far more than it costs, and not in the abstract hr/retention effects way, but in terms of concrete output.
More critically, these tools are now part of our tool set. That means when new opportunities come up, we can leverage them.
