People are rapidly learning how to improve model capabilities and lower resource requirements. The models we throw away as we go are the steps we climbed along the way.