I think diminishing returns applies regardless of whether it's improving itself through optimization or breakthrough new ideas. There's only so much firepower you can squeeze out if everything else were to remain stagnant.

Like if all modern breakthroughs and disruptive new ideas in machine learning were sent back to the 70s, I don't think it'd make a huge difference when they'd still be severely hamstrung by hardware capabilities.