> “The intelligence of an AI model roughly equals the log of the resources used to train and run it,” [Sam Altman]
Taking that at face value, it means we would have to invest exponential resources just to get linear improvements. That’s not exactly an optimistic outlook.
Even today's frontier models, without any further improvement, have incredible commercial potential. And even when improvements have diminisching returns in terms of result, the market shifts towards improved models in a supralinear fashion. So a linear improvement resulting from exponential investment might still net you exponential commercial return.
Also, the LLM space is a red queen environment. Stop investing and you are done.
All that said. IMHO Short to medium term breakthrouhgs will come from hybrid AI systems, the LLM being the universal putty between all users and systems.
The issue is, as it stands, LLMs are going to become a commodity. Could that happen and OAI is still a valuable firm? Maybe. Do I personally think OpenAI has the product prowess to pull it off? Nah, I think they are overly-concentrated with technologists (same problem Google has). Apple is still king when it comes to figuring out the ideal form of the product the buyer desires.
Apple can take a future OSS model and produce the winning product. That truth would be very bitter for many to swallow. Cook maintaining good relations with China could be the thing that makes Apple topple everyone in the long-run.
I wouldn't characterize that as “incredible commercial potential”. And especially in the case of translation, you save some of money, but also get poorer quality.
Speaking from experience I was recently in a tiny fountain pen shop in Sendai[0] where the owner doesn’t speak English and I don’t speak Japanese but we were able to talk for an hour or more about fountain pens and tomoe river paper alternatives using Google translate’s dialogue feature.
Maybe not massive commercial potential but it was pretty amazing and reminded me a bit of the Babel fish which use to seem like impossible sci-fi
OpenAI literally lost $12b last quarter, where is this "incredible commercial potential" you are talking about sir?? Is it monetizing the seven second slop memes? Where is the commercial potential you scam artist, stonk pumper?
> In its first decade (1998-2008), Google's total revenue was approximately $27.9 billion.
And now OpenAI has Google as their competitor. Besides, Google established a search monopoly by side deals, Android and browser push but they still lost the Asian market. Now OpenAi's got to overcome not only Google, Amazon and Microsoft but also Baidu, Deepseek and the other Asian and European competitors because nobody wants to lose the "AI race" - it's too risky. Without a monopoly, there's no high profit.
Even if you want to give OpenAI the benefit of the doubt by comparing it to other software primos, they're doing terribly. Google, Facebook, Apple, Amazon, etc. were profitable almost immediately after their founding. In the cases where they accumulated losses it was a deliberate effort to capture as much of the market as possible. They could simple hit the brakes and become profitable at will.
In OpenAI's case, every week yet another little-known lab in China releases a 99% competitive LLM at a fraction of their costs.
It's not looking good at all now or in the long-term.
I think he means resources as in compute cycles and the like. Those tend to increase exponentially for the same number of dollars in a Moore's law type way to intelligence should increase in an approximately linear way, something similar to a few IQ points per year.
And in his interviews he talks about the near vertical progress they're making. Of course nobody agrees but if those two are true, that's doubly exponential to keep pace.
> haha good one, so why haven't they done this yet? What are they waiting for? Let's see these super advanced "experts" with "specialized models"!!
I understand it's very easy to post ignorant messages in internet forums, but the answer to your question is yes, "they have done it" and it does result in cheaper training costs. See models such as DeepSeek-MoE or Mixtral.
Make openai a non corporate entity that belongs to the UN. It can't be profit driven. Fuck the economic pundits. This tech is beyond humans it needs to be done right. "All tech have been Enshitified to satisfy investor greed, sufficiently advanced tech can be called magic and these fools will enshitify magic also"
Taking that at face value, it means we would have to invest exponential resources just to get linear improvements. That’s not exactly an optimistic outlook.