One thing that is important to note is that to many this seems like an overnight success, but this guy has been cooking ChatGPT since 2017? I don't think at many points this seemed like it'd be this successful. It looked just like what google was going, some moonshot in some lab, that might in a few decades be a building block for real AI. And this was never cheap. Something to be said for making capital available for R&D and how some countries are doomed to lag due to aversion to this, governments and private parties alike.
Altman succeeded in reaching "the inflection point" on the PMF curve.
The other projects you allude to don't get their funding because most people who control budgets can only visualize the future with linear extrapolation.
You spend a LONG time in the low and flat part of your development before that starts to pick up, and up, and UP!
Homo Sapiens as a technology company were in the "fuck around with rocks" phase for 99% of their existence as a corporation. Agriculture in the final 1% of the development timeline allowed for industrialization in the final 0.05% of the time, which allowed for Large Language Models in the final 0.0001% of the time.
Had there been project managers and VPs in charge of deciding if our species was worth further investment, we'd have been canned in 800,000 BC.
It quite literally was. It had some of the fastest growing user growth ever. There was something like a million new users just days after ChatGPT launch.
Oh absolutely the AI has to have replaced or was all along our TV sama. Probably in the first season somewhere, with the twist being revealed somewhere in the 3rd or so. You'll see it plain as day during a re-watch.
Might be too obvious though, so maybe do a "never show the monster" move, let the audience stew on it and argue about it on the internet :)
not sure if they are very profitable and have no need for additional money, or they are highly unprofitable and require an end to their money bleeding.
Based on Microsoft's apparent losses on CoPilot, I suspect the latter. My initial research on ML usage suggested the thing wasn't profitable when commoditised which it well and truly is now hence why I've avoided investing in it (wisely so far). The energy cost versus capital cost versus actual utility makes it a net negative ROI for the technology supplier. The industry is currently living on some vain hope that hardware improvements will decrease cost enough for it to be profitable. With die and process shrinks starting to get problematic (3nm isn't going as well as people want it to) and increasing energy costs, cost reductions are unrealistic.
> not sure if they are very profitable and have no need for additional money
You’re going to make someone choke on their drink. The idea that a US tech mogul, and Sam Altman at that¹, would decide “you know what, I think I have enough money” is the realm of bad science fiction.
This account is six years old and posts frequently. Do you honestly believe I would know what Y Combinator is but not that it runs Hacker News? It changes nothing about the argument that Y Combinator’s goal is to make ever more money.
It seems that people would be willing to pay even more for the service for the value they are getting. And I think in near future such services will diversify. Some becoming very expensive and providing high value. Which will further increase the power of money. This time also connecting it with access to knowledge. For now, public knowledge, but a deal where say Amazon AI has access to all digitized books and provides spotify-like earnings - yes in multiple meanings of that phrase - seems like a possibility.
Remember the section of the OpenAI Dev Day keynote when Satya Nadella from Microsoft got up on stage? One of the main things he was talking about was providing compute for OpenAI. He mentioned how aggressively OpenAI was pushing forward and that the workloads were unheard of.
Last time I talked to our Azure reps, the technical folks seemed to be saying (or rather wanting to without explicitly saying so) that basically all present and near term GPU capacity (of recent gen) is either being consumed or will be consumed by OpenAI. Or in other words there is literally not enough GPUs on the planet (reasonably available) for MS to cover both OpenAI and Azure GPU loads without affecting the other. We basically couldn’t get them to commit to any price or delivery schedule if it involved the same hardware GPT needs.
We are a $110B+ company btw and a strategic partner of MS, not some random shop. They’re not supposed to just say “no.”
I can't find it, but there's an old quote along the lines of "it's easy to reach a net worth of a million dollars, all you have to do is start with two million and lose half of it".
This is a classic that exists in many variations. The one I've heard most often is "the way to make a small fortune in photography is to start with a large fortune."
and is extremely slow - almost useless for me comparing to previous versions. I would still prefer to choose explicitly GPT-4 model that doesn't browse and don't do image recognition.
Now too often it goes to searching the web even if sometimes I don't need up to date information e.g. about GDP of some countries I just only need some ballpark stats.
> I would still prefer to choose explicitly GPT-4 model that doesn't browse and don't do image recognition.
I have disabled it in settings. Which only decreased usage of it. Which is ridiculous, especially that you can avoid using e.g. "browsing" by explicitly stating it in the input.
Free cgpt users and low tier API users are downgraded opaquely to lower models when demand is high.
It's irritating because I have over 4000 API chats in a DB that I like to analyze, but I cannot now know for certain which model was actually being used.
Ohh so that explains why I saw `?model=text-davinci-002` appended to my ChatGPT URL the other day. I remember it was pretty slow to load then, too, so that's probably why they rerouted me
Yeah it did but it was annoying. It kept blocking my account on the iOS app for no reason (really, not even a hint of something against the tos), they modified it so you could ask only 3 more follow up questions etc.
Possibly ridiculous question, but does GPU lifespan decrease with more intense use?
I could imagine, if so, that they bought some hardware, which needs replacing sooner than they thought, and the running costs are catching up with them.
> but does GPU lifespan decrease with more intense use?
In principle, yes, could do (solid state electronics do wear to some extent) but in practice data centre GPUs will be specced for a low failure rate under constant use.
Mosfets have lifetime depending on the temperature, so yes. But on the other hand vendor can install components which won't fail for some long time even under max load. If I remember correctly this was the reason for the "New World destroys GPUs", because some vendors didn't read the Nvidia spec carefully and installed less robust mosfets, so the max load (which happened to be New World game) pushed them over the expected lifetime fast. But with proper design this seems not to be a case.
I have no idea how many tokens/second OpenAI gets[0], but 100 M users, each generating an average of (to pick an arbitrary number for a Fermi estimate) 1000 tokens per day, is 1e11 tokens/day or ~1,157,407/s, which could be anywhere from 36k severs to just 830 depending on which value[0] I use as the anchor to guess how many tokens/s their specific servers can generate with their specific models.
At the upper end, they may have bought about 8% of all H100s currently in existence. The Microsoft investment (1e10 USD) correlates better with the higher range than the lower range of my estimates for how many machines are in their cluster ((36k * H100) * $30k/H100 ~= 1e9 USD).
[0] a quick google said 30/s in the automated answers section (which is citing a May post on Reddit), while one of the links says ~300/s on an H100, and the Llama.cpp github page says ~1400/s on an M2 Ultra (but for a relatively small model).
I would imagine at such scale this abstraction starts to fail. Eg say they wrecked Microsoft's GPUs, MS probs can't just provision more somewhere else at a blink of an eye.
They certainly handle google scale compute. It's a scale even google couldn't handle at the moment uncontrolled. Hardware for this art of compute is very limited and not remotely comparable to google search
It is indeed comparable given that people here are talking about replacing Google with ChatGPT as a so-called 'search engine'. Even in that case, ChatGPT search queries are far more expensive than a Google search and requires more hardware to scale regardless.
Google's scale of usage is 8.5 billion searches per day. For OpenAI, the costs for that scale will just skyrocket and OpenAI cannot handle that which explains why they are already pausing new sign ups even when they have a login wall.
So it would be even worse for OpenAI had they opened ChatGPT up without a login just like Google doesn't require a login just like most search engines.
Let's not act like Google Bard isn't behind curtains. They face the same scaling issues like Openai/Microsoft. People are using search in a very different way than a LLM. Openai isn't simply replacing a search catalog to websites, it's replacing the internet as we know it so i don't see how this can be compared. Most people on earth won't have nearly unlimited access to this technology at the moment because of hardware limitations that all providers in this game face but those who have are at an advantage.
Bottlenecks either because they switched all Plus users to GPT-4 Turbo, which isn't a production model, or that GPT builder is a fine-tuned model of GPT, and needs more bandwidth, or that they actually have reached capacity and need to scale their code to more servers allocated to each GPT-4 Turbo and the fine-tuned builder model.
Is that true? I'd expect the turbo version would reduce their workload, because it (probably) uses less gpus. AFAIK, it's still in preview, only available via api.
More likely the publicity around their dev day has increased signups beyond their capacity.
Is that true? I'd expect the turbo version would reduce their workload, because it (probably) uses less gpus. AFAIK, it's still in preview, only available via api.
Maybe one day their inference could be running on photonics hardware!
Or imagine for the code analysis they replaced phyton running on kubernetes with wasm engines. would need a small wast to wasm compiler and interfaces for simple io and plotting, but many analytics tasks could probably run much faster.
I think the inferencing is using many orders of magnitude more compute than the occasional analytics tasks. But I assume the containers do need some significant resources.
As far as making analysis lighter weight, I think that something like you say will eventually be the way they go. For my own agent hosting of agent-written code, I moved to Rust and have been incorporating Rhai scripting which is vastly more efficient than containers.
what do you think about Sam Altman's policy requiring all users of GPTs to subscribe to ChatGPT Plus, not just the developers of GPTs. This requirement applies regardless of whether the Plus features are utilized in the bots' creation? And now he suspends new Plus accounts so people who promoted their GPTs with no promise of ever seeing money from it, cannot even have their audience use it if they wanted to.
what do you think about Sam Altman's policy requiring all users of GPTs to subscribe to ChatGPT Plus, not just the developers of GPTs. This requirement applies regardless of whether the Plus features are utilized in the bots' creation? And now he suspends adding new users, so allot of promotion of GPTs to people who cannot use it even if they wanted to!
"Unlike his infamous and original bearish bet against subprime, Burry's latest attempt to time a market crash has crashed and burned, because according to the just released 13F from Burry's Scion Capital, the $1.6 billion notional in puts on the SPY and QQQ have been liquidated."
(Not mine, stole it from the Twitter thread. Just thought it was funny.)