It also tells us the genie is out of the bottle not just in the form of open weights being widely available, but in the form of the text corpuses coming from the existing model. The claimed low cost of Deepseek's training is partly enabled by the availability of all that synthetic data created by the first generation models trained and developed at much higher cost. When the Soviets got hold of the nuke plans, they greatly reduced their development costs by primarily by not having to redo all the experiments that led to dead ends. What's amazing is that time it's different; nobody needs OpenAI's secret sauce anymore, just enough data - some of it happily supplied by ChatGPT itself, and they can experiment with different architectures and either get tolerable results with an architecture already in textbooks, or greatly improve efficiency by innovating.
