There are diminishing returns from compute time but it looks like even though they are diminishing there's still a fair bit on the table.

Though my guess is that GPT-5 will be the last model which gains significantly from just adding more compute to the current transformer architecture.

Who the hell knows what comes next though?