The iPod didn't stop growing. It turned into an iPhone - a much more complex system which happened to include iPod features, almost as a trivial add-on.
If you consider LLMs as the iPod of ML, what would the iPhone equivalent be?
Large multimodal models. Maybe using diffusion transformers. Possibly integrated into lightweight comfortable AR glasses via Wifi so they can see everything you can, optionally automatically receiving context such as frame captures. Optionally with a very realistic 3d avatar "teleported" into your space.
Probably the kind of intuitive AI assistant you can find in a lot of science fiction works. A virtual assistant that can search the web for you, write/proofread an essay, analyse social situations, and enrich the right bits of your personal information for you. Basically a smart assistant integrated into a vr headset.
I'd rather consider the current generation of LLMs as the first iPhone (or maybe flip phone, depends on where you stand), given that hundreds of billions of dollars in private money is already going into developing them.
He made an analogy to discuss the business case for scaling an iPod, not whether or not new products and services would be invented. My iPhone still only has about 64gb of storage.
If you consider LLMs as the iPod of ML, what would the iPhone equivalent be?