There are a lot of gains (both cost and performance) to be made here, but it's not "low hanging fruit". This is not the scaling of yesteryear, and the importance of design/testing/reliability and yield are really going to rise as the number of parts in an already complex product explodes.

It would be so cool if we got to the point where startups and OEMs could design their own

You absolutely can already using OSS tooling, Verilator, Chisel, OpenRoad, Yosys.

Chiplets are first a design methodology and then a burgeoning standard.

We need AI accelerators with at least a dozen DDR5 memory channels.