To be fair, training is a lot more intensive than inference. Though if open source models are any indication, the big issue is actually VRAM requirements.