One thing I don't understand well is how much computation using GPT-4 takes. Some of these discussion remind me of Bitcoin as a global payments processor: sure, it can work, but it's doing a tremendous amount of computation and the maximum rate of transactions it can sustain is pretty low.
I know it used a huge amount of energy / GPU cycles / time to train, but now that the weights are computed, what's involved in running it? I know the model is huge and can't be run on an ordinary developer's machine, but I believe requests to it can be batched, and so I don't really know what the amortized cost is. Right now, this is all hidden behind OpenAI and its credits; is it running at a loss right now? How sustainable is using GPT-4 and beyond, as a day-to-day part of professional life?
I know it used a huge amount of energy / GPU cycles / time to train, but now that the weights are computed, what's involved in running it? I know the model is huge and can't be run on an ordinary developer's machine, but I believe requests to it can be batched, and so I don't really know what the amortized cost is. Right now, this is all hidden behind OpenAI and its credits; is it running at a loss right now? How sustainable is using GPT-4 and beyond, as a day-to-day part of professional life?