Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Nearly half of Nvidia's revenue comes from four mystery whales each buying $3B+ (fortune.com)
236 points by mgh2 on Aug 31, 2024 | hide | past | favorite | 234 comments


According to Observer they are: Microsoft, Meta, Google, and Amazon.

Other big buyers area: Oracle, CoreWeave, Lambda, Tencent, Baidu, Alibaba, ByteDance, Tesla, xAI.

https://observer.com/2024/06/nvidia-largest-ai-chip-customer...


"Who could it be... Hmm... Such a tough nut to crack..."

(Even without a report on this it would be obvious)


Meta can be confirmed as one since they’ve literally mentioned their infra investments and Billions in capex increases until the end of 2025 in every earnings call this year.


I guess Apple is using their custom silicon?

That was a major payoff for Apple - I wonder if any of the other fangs will actually be able to follow suit.


Apple uses TPUs on Google Cloud Platform. https://www.cnbc.com/2024/07/29/apple-says-its-ai-models-wer...


And a weird deal with OpenAI (which I think would show up as Microsoft for the actual physical hardware). https://openai.com/index/openai-and-apple-announce-partnersh...


for training, but for the interference apparently they use their own chips


Except in cases where responses are outsourced from OpenAI. All the ChatGPT-based inference likely happens on Nvidia hardware too.


Is there evidence that Apple is training a model large enough to require a huge amount of compute?


https://arxiv.org/abs/2407.21075

AFM-server was trained on 8192 TPUv4 chips

Someone more versed can say if that is huge or not.


It is a far larger scale than most high performance clusters offer.


At one point over 50% of their server revenue came from hyperscaler. Which is exactly the same as those four listed.


Utterly surprising


Where is Apple in all of this?


Apple historically dislikes NVIDIA and I they would likely rather use their own in-house chip team. They also rely on it by virtue of using OpenAI in upcoming iOS release.


They don't like the high margins? :P


TBF Apple dislikes any other company they work with. They're seething with rage any time they cannot do something in-house.


I wonder if the split happened with jobs or after jobs? I thought jobs was good at relationships with everyone else in silicon valley (intel, ati, nvidia, even microsoft)


Apple dropped Nvidia after a few years of Nvidia falsifying thermal specifications on GPU chips.

It drove apple crazy both with high failure rate of MacBooks where the GPU was desoldering itself and general problem of a hot as fuck bottom. Nvidia refused to pay out for damages to Apple as well from what I recall.


To add to that, NVIDIA tried throwing OEMs under the bus when the issues cropped up. It wasn’t just Apple who were affected.

Then a few years later they made up but NVIDIA didn’t want to partner on drivers, so they had another rift


IIRC it was with Jobs. Apple wanted to develop their own drivers for their chips from ground up, and NVIDIA was very secretive of their tech, so things went south.


> NVIDIA was very secretive of their tech

Oh the irony for Apple to dislike others being secretive...


On their own right, they contribute more than many other companies, though. Their kernel is open source, they have given their secret sauces like Grand Central Dispatch away, allowed complex technologies like mDNS (Bonjour), AirPrint, multipath networking to be implemented freely and used widely in a vendor agnostic manner.

macOS is 1000 times better for talking UNIX systems than Windows and is POSIX compliant.

Lastly, they are not hindering the development of Asahi Linux, and did nothing when their devices were reverse engineered. On the contrary, they left a couple of ways open for Asahi guys to boot their distribution directly.

They are not the band of saints, but they are not the underhanded evils like a couple of others.


They're shipping queries off to ChatGPT so I guess this ends up as nVidia cards on Azure?


Apple hates Nvidia so wouldn't buy directly from nvidia.


Kind of embarrassing for Google to be on that list, no? Shouldn't their in-house TPUs be cost-advantageous for them?


No, because GPUs are not only for AI. They are MATMUL machines, and MATMUL is useful way beyond AI and tensor applications.

Some of us use them at double precision mode.


Yes, but demand for these chips went through the roof because of AI. If Google is on this list it's because they're using them for AI, not because they've got a secret project rendering an insane number of 3D images or something.


Everything from material simulation to weather forecasts use GPUs very actively and effectively, for a long time.

There’s a whole world using GPUs to accelerate things.


Right, I'm not arguing against that.

I'm saying that Meta and Amazon and Microsoft are all buying these chips in insane numbers for AI—their usage for all other types of GPU activity is at least an order of magnitude less. That's why Nvidia skyrocketed to become the most valuable company over just a few years.

For Google to be on that short list of whales would either mean that they for some reason have a much larger demand for GPUs for non-AI purposes than any of the others have for AI purposes (doubtful) or that they're using GPUs for AI.


Those are likely for Cloud, used by clients.


Google has addressed this. They offer GCP as an AWs alternate.

They’d rather offer their clients what they need than push them on to their own products.


I understand why the state of affairs is, the point is that it's pathetic. Google, an AI hardware manufacturer[0], has to eat a direct competitors not in substantial margins in order to offer their customers, external and internal, a viable product.

[0]and supposed software powerhouse.


Microsoft has Linux on Azure.

Amazon doesn’t force everyone onto graviton cores, and offers competing cores.

It’s just the nature of business.


Okay, I mean I feel like they’re not that mysterious. Like, there are probably only five or six candidates.


Companies like Coreweave who lease accelerators may make this analysis less straight forward than it appears.


Even less straightforward since Nvidia funds Coreweave.


> Although the names of the mystery AI whales are not known, they are likely to include Amazon, Meta, Microsoft, Alphabet, OpenAI, or Tesla.


Why mention Microsoft twice?


The departments aren’t talking, so they accidentally made two orders.


I’d note Microsoft needs OpenAI a hell of a lot more than OpenAI needs Microsoft. I’d actually pivot that to be why mention OpenAI twice.


How so? As far as I can tell, Microsoft has a large equity interest in OpenAI, and OpenAI has a lot of cloud credits usable on Microsoft’s cloud. I don’t think those credits are transferable to other providers.


The value in the proposition is OpenAI IP. Money and data centers are commodities easily replaced, especially when you hold the IP everyone wants a piece of.

The arrangement is mutually beneficial, but the owner of the IP holds the cards.


OpenAI doesn’t operate without the enormous amounts of funding MS gives it.


I think a lot of institutions and people would love the chance to give them money.


But how many of them have hot data centers to offer? Google is a direct competitor, so Oracle or Amazon are kinda the only other two big options to offer them what MS is right now.

If MS drops OpenAI, it's not like they can just seamlessly pivot to running their own data centers with no downtime, even with pretty high investment.


A relationship that’s mutually beneficial needn’t be symmetric. Microsoft’s relationship is fairly commoditized - money and GPUs. OpenAI controls the IP that matters.

I’d note that the supplier of GPUs is Nvidia, who also offers cloud GPU services and doesn’t have a stake in the GCP, Azure, AWS behemoth battle. I’d actually see that as a more natural less middle man relationship.

The real value azure brings is enterprise compliance chops. However IMO aws bedrock seems to be a more successful enterprise integration point. But they’re all commodity products and don’t provide the value OpenAI provides to the relationships.


And regardless of who the 4 are, the two others in that list of six candidates are most likely not too far behind.


The most interesting thing to discover would be if one of them is that far behind, because they're succeeding on their own/someone not-Nvidia's silicon.

The public in-house projects that I'm aware of (but as far as I know haven't fully replaced demand for Nvidia GPUs) include:

- Google's "TPU" (in production, publicly rentable)

- Amazon/AWS's "Trainium" (in production, publicly rentable)

- Meta's "MTIA" (in production)

- Microsoft's "Maia 100" (I'm unclear on their status)

- Tesla's "D1" (I'm unclear on their status)


It's not just the hardware, it's the software stack too and my understanding is that they aren't very good. Even TPUs aren't great if you aren't either (1) doing something extremely standard and a little bit old (e.g. not forefront of research and the stack has already been optimized for your model) or (2) in Google with access to the people who build the stack.

Maybe it is working for Meta or Tesla where things can be vertically integrated, but for the public clouds, they have to buy NVIDIA for their customers.


MTIA will be for inference initially. Another to add to the list is wafer maker Cerebras

https://www.forbes.com/sites/craigsmith/2024/08/27/cerebras-...


Are these distinct architectures, or is it an ARM situation where nearly everyone is gluing the same IP cores together in slightly different configurations?


I'm not sure about all of them but Google's TPU is custom to them and not shared architecture.


They are distinct architectures, but mostly do the same thing. Pretty much all of them have a few small control cores that run matrix multiply and vector reduction units. The instruction set on all of them is different, but the broad strokes of the architecture are the same.


Definitely not an ARM situation.


xAI 100K H100 cluster would be ~$2B

Is this trailing year NVIDIA sales, or order book ?


The numbers in the article? They're neither. They're the revenue for just Q2, not the full trailing year.


Does TSM have the capacity to scale up anything new right now?


Let's name them, and why?


Anyone building the largest of LLMs including Alphabet, Meta, OpenAI, Anthropic


I doubt Anthropic builds their own GPU datacenter.

They might buy some, but I think that Google, Meta, Microsoft and Amazon and will be the ones buying in large batches to enable companies like Anthropic (and themselves) to scale up to world wide inferencing demands, as well as generally offering the most efficient GPUs to their customers.


Think they're renting GPUs from the cloud providers?

Very plausible. I'm not sure at which point it makes economic sense to buy the GPUs and build out the infrastructure to continually be training something like Claude.


> Think they're renting GPUs from the cloud providers

It's a major reason why they raised with Amazon [0]

There are actually a LOT of other large companies that participated in Anthropic's round but haven't announced it publicly.

[0] - https://www.aboutamazon.com/news/company-news/amazon-anthrop...


They train on GCP now.


They also raised with Google, interestingly.

https://www.wsj.com/tech/ai/google-commits-2-billion-in-fund...


Yep! You're right! I think they were initially using AWS though.


I think you should probably add all large cloud providers. Amazon and Microsoft should be in the list.


And we know OpenAI uses Microsoft for GPUs. My guess is Anthropic is similarly not owning their own data centers; didn’t they get a bunch of money from google? It’s probably being spent there.


They use AWS.


No they don’t.


Care to substantiate?

According to Wikipedia:

In September 2023, Amazon announced an investment of up to $4 billion, followed by a $2 billion commitment from Google in the following month.

They probably use both. Almost certainly most of this is credits in their cloud platforms.


Oh, I would not forget governments. The NSA basically paid for Kepler with a single purchase...


I believe the GP refers to Google, Facebook, Amazon and Microsoft.

And while yeah, that's probably them, I'd put a non-trivial chance of some government intelligence organization to make it into the top 3.


> I'd put a non-trivial chance of some government intelligence organization to make it into the top 3

Most likely DoE. TLAs purchase indirectly (or use other federal agencies in the DoD as a front)


ANd possibly some less-known companies, as fronts for the Chinese.


> less-known companies, as fronts for the Chinese.

Not at that size. That is VERY on the nose sanctions evasion.

Such sanctions evasions tend to use multiple smaller parties doing purchases and then reselling.


I’m guessing a massive amount is for inference for whatsapp, and the original goal was making for relevant instagram - and of course the massive Llama model training - my guess is Facebook is a relatively small component of Meta’s overall use of GPUs. Feed recommendations? (unless you were using facebook as a holder for Meta?)


It’s absolutely not WhatsApp. It’s their recommendation engines. They’ve publicly stated they’re buying enough GPUs to have the spare capacity to train another “reels” sized product for when the opportunity emerges.

(They absolutely use it as a holder for Meta)


I think Meta has been pretty transparent about their GPU purchases[1]. 350k H100s should go pretty far into the billions.

https://blogs.nvidia.com/blog/meta-llama3-inference-accelera...


Two questions that need answering:

1. Are chatbots going to get much more effective than they already are? It seems like all the major players are plateauing and the different models are becoming commoditized. That doesn't bode well for sustainable GPU sales. Also if the hallucination problem can't be solved, it's not clear that this generation of AI will ever be deployable at scale.

2. Are there genuine at scale use cases for AI outside of LLM's? Autonomous navigation seems like a major one, but I'm not sure how close that is to production ready. I know drug discovery and other applications are talked about, but not sure how much GPU consumption they can realistically generate. As we leave the novelty phase of the adoption curve, it's clear that a lot of the use of the image generators was unsustainable experimentation. My personal experience has been, a year ago my friends were creating tons of images but now we hardly do at all.


> Are there genuine at scale use cases for AI outside of LLM's

Assuming that "outside of LLMs" means "outside of text processing".

Yes. Robotics. Imitation learning with LLMs is working surprisingly well. It will require a lot of investments in data and training to get to a practical state, but all the early signs point that new revenue streams will be unlocked in Robotics.

One limitation that still stands on the way is the inference speed. My estimate is that we need ~10k tokens/sec prompt processing speed to get these smart robots working reasonably fast. We're getting there for 8B models (Groq & Cerebras silicon), but these 8B models are too dumb (especially, after being finetuned on robotics data), and 70B models are still 20x slower than practical.


Going to be ugly when the AI bubble busts


I wouldn't bet in bulk computation not being in high demand for the foreseeable future.

If the AI bubble bursts, people will use the available GPUs for something else.


> If the AI bubble bursts, people will use the available GPUs for something else.

Yes, of course, but that just means that this bubble would be basically identical to previous capital intensive bubbles. For example, there was a railroad bubble in the 1800s, and a massive telecom bubble in the late 90s. These bubbles popped, resulting in massive corporate bankruptcies and failed companies. But the infrastructure they built (miles and miles of railroad and dark fiber, which has since been lit up) laid the foundation for huge economic development shortly thereafter.


The railroad built during the bubble in the 1800s, like 90% of it is decommissioned. It served no sustainable economic benefit, as most of it was last mile railroad that quickly got consolidated into trunks.

If the US had maintained and kept the rail it built, it wouldn’t have the poor infrastructure it has right now.


The relevant part is that no, the steel industry didn't break when the railroad bubble burst.

Nvidia is not the train company on that scenario.


I hope the bankruptcies start soon so I can buy me some H200s for cheap on eBay


I'm confused by this sentiment I've seen repeated by some.

AI/LLMs are radically expanding my abilities, and as I adapt to this new power, I'm using it more frequently in everyday life.

Sure, Nvidia stock may be overpriced, but AI is empowering. I can't imagine not continuing to expand its use. As its abilities expand, I'll use it even more. I will have much further use even as a few bugs are fixed and integrations become more frictionless.


Probably because not everybody is feeing this productivity boost. AI made me a bit more productive, yes, but not by that much. Seeing you call it a "new power" is not relatable, so it may reinforce ideas that it is a bubble.


Right but I feel similarly about excel/sheets and powerpoint. But they make almost every office worker a bit more productive, so it’s a good market.


I think these can both be true. If AI makes people be 3% more productive overall, cumulatively that's a huge improvement, but on an individual level it may feel undeserving of hype.


> Seeing you call it a "new power" is not relatable, so it may reinforce ideas that it is a bubble.

Whereas what it could be reinforcing instead is that some people are better at "using AI" than others.

When I was young, I always saw how my parents never really "got" new technology that I was using all the time, like the internet. Many young people think about it and are sure it won't happen to them. I'm sure many on this technophile site think so.

And then a new technology like AI comes along, some people find ways to be incredibly productive with it, but a very widespread sentiment is that they're... lying? Mistaken? Not very good at their job so it helps them more? The number of excuses people have for "keep this new technology that I don't know how to use away for me" is pretty crazy.

(And I say this as someone who is probably not on the "cutting edge" of AI usage, compared to others I see.)


That seems like saying "web advertising never shows me what I want to buy"

Maybe it is not for you. Maybe it is for people asking AI questions about you. (or chemistry or gold prospecting or legal documents or ...)

the eric schmidt talk made it seem like better hardware led to better results and there was a race.


People got burned by crypto, they promised that it would replace Fiat money but all that happened is that they lost all their money investing in NFTs or Web3. So now they are jaded against any new hyped technology, and have no interest in investing, and are actively hoping it fails for FOMO reasons


You can’t throw a stone very far without running into an IRL businesses or ATM that takes crypto (In my small town there are many), congress is writing laws to legalize it, a presidential candidate is running on it, and the Fed is creating a “coin”.

When businesses stop accepting dollars and your employer starts compensating you in crypto will it stop being a “scam” or will the goalposts move again?


That is wonderful that there are Bitcoin ATMs. The profit generated by the owners of those ATMs isn't a scam, it is real revenue.

The scam is comparing some ATMs to what is happening in AI. Trillions of dollars are going into AI and actually useful things like self driving cars are coming out.


Curious what year do you imagine that will be?

(I don’t expect it to see it in my lifetime.)


Eh, anything AI is clearly in a massive bubble.


The real world actual usage of neural network software is increasing. People are finding uses for it. They are replacing or augmenting search and lookup. Translation, proof reading, and copy writing is actually useful. AI assisted coding is exploding and people are paying for it. People are using it for design and product prototyping. They are amazing at data analysis, which is very useful for enterprise and finance (lots of money here). It's used for medicine research and drug discovery. It's used for legal document analysis. So much actual real world value here. So much gdp impact possible.

Most cryptocurrencies are just straight up scams. Some people are getting some usage of Bitcoin as a store of value and for cross border transactions. This is increasing slowly. Stablecoins also have some usage for store of value and cross border transaction. They are also used for trading and arbitrage, which you can argue about whether that brings value to the world. The rest of the crypto market is struggling to find an enduring use cage. I'm saying this as someone who is marveling about Ethereum and Solana, but I don't value trading immaterial NFTs. Ethereum and the like are struggling to do anything that reaches into the real world. None of the cryptocurrencies outside the top 10 have found any real world use case that people care about.

So AI != crypto


I suppose people might be wondering what happens when this small number of big players finish their buildouts.


This. AI itself might be here to stay, but how much revenue growth specifically is left?


It feels suspiciously like it’s 1999 all over again.


I totally agree, but I feel like some comments are making the mistake of stating "this bubble will pop, which means it was all smoke and mirrors to begin with".

The dot com bubble popped, but it's not like the Internet technologies that were launched then (and companies like Amazon and Google) weren't hugely impactful on all of society since then.

I think the AI bubble will pop, and while I think there is a lot of nonsense hype about AI I still think AI's societal impact will only grow.


No one who is saying the bubble will pop thinks there is nothing behind it. That’s the definition of a bubble: you always need soap and water to make it, but soap and water are commodities, not this special unicorn that will change the world.


Beanie Babies, Tulips, NFTs, Web3 tokens were all obviously not going to change the world. The bubble was pure emotion and greed. All the cash inflows were speculation.

Nvidia made 18 billion in profit last quarter, and expects to make 20 next quarter. That isn't speculation.


I mean, how much money did the NFT companies and Tulip vendors make? Nvidia isn't OpenAI. They're selling products that the bubble is built on, not the bubble itself.

How much money is OpenAI or Anthropic making? Because that's what people are thinking is speculative value.

My position has always been that Gemini/ChatGPT/Claude are all pretty great at a cost of Free, and grow increasingly questionable past that. My work is already limiting how many ChatGPT users we can afford with their price increases, and I'm pretty sure OpenAI is still not profitable. If ChatGPT is $50/month as a breakeven cost for them, how many people/companies will buy it then? Most jobs I've been at won't pay for JetBrains licenses that cost way less per head.

I feel like the best comparison is something like Uber or AirBnB where it's easy to be excited about it when all the services are crazy discounted by free VC money, but when they have to start turning a profit, they're back to actually competing with other tools.


The cost to run models of a specific quality goes down by about 90% a year. So the unprofitable $50/month cost becomes profitable in about 6 months.

But the big deal isn't OAI being a profitable company. The big deal is that GPT6 will be 100x more useful in doing productive work.

Tulips did not have cash flow like this. It was only people selling to speculators who hoped to sell again to another speculator.


> The big deal is that GPT6 will be 100x more useful in doing productive work.

Citation extremely needed. There's a lot of people and companies downstream of OpenAI speculating on that 100x that are gonna be in a lot of trouble if it's even just 10x, let along 5x.

Again, not saying that none of this has any value, just that the value may well never live up to the cost. Uber's not a worthless company or service, but they're far from the values or profits they were pitching 10 years ago.


I would point you to tow pieces of data.

1. Drive a new Tesla with the latest Supervised FSD and measure how often you have to intervene to stop a crash.

2. Go back and look at your own expectations around AI two years ago. Did thing progress the way you expected or did they progress further?


1. You are comparing 2 very different things. GPT6 is a generative LLM. Tesla's FSD is machine learning.

2. I have no expectations for 'AI' because the term is a nonsense label. I have followed and been excited by machine learning for a good number of years, and my expectations of progress were pretty much on par. The progress with LLMs has taken me a little by surprise, but I am also cognisant that their progress is being massively over-hyped presently, not least by ppl who call them 'AI' and then, even more foolishly, go on to talk about 'AGI' (a nonsense upon a nonsense).


I agree that AGI is a meaningless term.

I'm intentionally including FSD and LLMs under the same category of technologies that will have a huge impact. The point of this thread is that the demand for inference is going to skyrocket because AI is going to get a lot more useful.


Putting aside my (trenchant) philosophical issues with the term 'AI', I also don't think just pragmatically that it's a good categorical label.

We both appear to agree that Machine Learning is a very powerful technology that will have huge impacts. Machine Learning requires (and will continue to require) a lot of compute and thus large costs but will also, almost certainly, produce great profits in some domains (FSD being one).

It's a lot less clear to me that LLMs will 1) continue to require lots of compute beyond the short term (languages can get close to being 'solved') or 2) that LLMs will generate substantial profits because a) the model can escape capture from a monopoly player far more easily and b) while useful for translation, pulling summarised data from a corpus, recognition of voice commands, etc, none of these applications actually make for the kind of profound impacts that ML is capable of, because none of them transcend human ability like ML has the power to do.


Reasoning and MultiModal are emerging out of the larger LLMs. That opens up more use cases, which then drive demand for inference. And that also drives demand for more research. It is hard to say exactly which use cases are going to be huge in a year but it seems very likely that more use cases will open up given how widely you could apply even a small amount of visual reasoning with robotics.


1. The friend I know with FSD has had it nearly kill him twice in the last year, but it does seem notably better, but in the sort of incremental way I'd expect. They keep it more as a novelty than a functional service.

2. If anything, GPT4 has turned out to be less of an advancement over 3.5 than either OpenAI was claiming and what I'd expected. 2 years ago, people were all but promising AGI by now. Even the folks I know working in the GenAI space are telling me they're using Copilot/ChatGPT less now than a year or so ago. My work has actively cut back on spending in the area and investors have been asking our board questions to make sure we're not overinvesting in it.

I want to be clear, I'm not a doomer at all about this. I use these tools a fair bit and find value in them. But the value that GPT3 and 3.5 brought to me versus what GPT 4 has brought certainly isn't 100x. GPT4 isn't even 100x better than me using Google Search most of the time.


More like the AI winter from the late 80s.


If by "AI winter" you mean a period where AI will continue to be used for semantic search, moderation, translation, captioning, TTS, STT, context-aware grammar checking, LLM, and audio/image classification, then yes, it would be an "AI winter" where AI is used everywhere.


I meant specifically the time in the late 80s when investment in AI collapsed because it was overhyped and caused the downfall of Lisp Machines. The AI field itself kept moving forward, but investment and grant funding was cut to almost nothing for a long time. It took a long time for the field to get to where it is now, but the hype cycle has been going back and forth for decades in AI.

https://en.m.wikipedia.org/wiki/AI_winter


I know well what the 80's AI winter is, the next one will have AI used everywhere if it's going to happen.


No it doesn't. The economy then was actually good.


And housing was cheap and plentiful.


Not really compared to 1974, though. And I am sure in 25 years time we will be complaining about how good we had it in 2024.


If 2024 is the high point for a quarter century, we really are going to rock bottom as a world.


Don't think of this as the worst year of your life. Think of it as the best year of the rest of your life. :)


Everything is relative.


It's a shame all this compute is being built and none will trickle down. It would be fun to hack on this stuff as a hobbyist once it's sold for peanuts.


I assume used V100/P100s are on eBay. Go buy them and report back.


How is it going to burst exactly? Are ChatGPT and Google Assistant going to suddenly stop working?


growth stops, investments stop, projects and orderds get cancelled, consolidation happens, unused stock shows up on the secondary market which puts a downward pressure on unit prices of nvidia datacenter GPUs


I’s going to be nice to get cheap gpu.


They might even put reasonable amount of RAM on reasonably priced models... Why can I not get 16GB on some 700€ gaming cpu... I can get CPU+Mobo+32GB ram for around same... I just hope this intentional kneecapping ends so I can get something that can be used for a few years.


If you can't use a "reasonably priced model" GPU for "a few years", I'm really confused as to what you're doing. I know people still using 1080's and 1080Ti's and playing pretty much anything they want to, and I only just upgraded from a 2070 Super to a 7800 XT (with 16GB of RAM on it, even) this summer.


Margins on datacenter GPUs will probably always be better than consumer. As long as thats the case they need to segment to stop datacenters from using consumer products so I've got a feeling that you will never be able to buy a consumer nvidia product with a reasonable amount of RAM. Maybe intel will release one to get some hype for their gpu line?


Does that mean that datacenter hardware might be a cost wise option for making a home lab PC soon?


A GPU is an accessory to the real product you’re buying: a driver to interface with your software. Datacenter GPUs have drivers that are woefully inadequate for gaming.


Probably better to just throw your workload on the cloud unless you're using it close to 24/7.


I’m just thinking that maybe trading a decent used sedan for a slightly shinier ARPG isn’t the wisest move I could be making.


Is it possible to use an H100 as a gaming GPU? That would be neat to see.

Oh. It's been tried: https://www.pcgamer.com/nvidias-ultra-expensive-h100-hopper-...


I don't think it would work as well as a 4090 that has almost as many shader cores, and quite a few more cuda cores than h100.

Seems the pros of an h100 over a 4090 are: much higher vram, much faster vram, technologies like nvlink available, and a focus on lower precision performance more useful for ML (as opposed to 4090s focus on fp32).


I assume the consumer GPU and data center products have minimal overlap. If NVidia never sold another server product, would that really impact consumers all that much?


There isn't infinite production/packaging capability, and they're going to prioritize the customers willing to pay more for the chips they get out of a wafer. Another aspect is that the chips are different between compute and consumer, as opposed to something like a Zen chip where it can be used in either Epyc or Ryzen.


I have no idea what I would do if cheap GH200s started showing up on Ebay. They would probably need some crazy cooling and interconnect to get working. I guess it would be the ultimate "localllama" machine.


I don't know, when bitcoin crashed there were a flood of clapped out wrecked GPUs on the market but nothing that I'd risk buying.

Same'll happen here.


is it a bubble? or is the singularity? (only half joking)


It's going to rock for gamers.


There’s no such thing as an AI bubble. That’s like saying, going to be ugly when the car bubble bursts, back in the early 1900s.


So, I feel like your arguments is "AI is useful, like cars, so there won't be a bubble"; but like, I think we must all agree that the Internet is useful, and yet there certainly was the ".com bubble". We've occasionally had real estate bubbles, and I do in fact believe there was a car bubble in the early 1900s during the 20s?



Clickbait title. Even the article cites the number from Jensen’s interview.

https://youtu.be/NC5NZPrxbHk?si=8uQ4zdMU02f4X1Hc (at 1:41)

Hyperscalers & Meta.

(Corp speak 101: Hyperscalers = AWS, GCP, Azure)


Pretty good indicator of the nonsense hype bubble of AI!


It's not like AI hasn't been delivering during these past 3 years, and it's just getting started.

There's no one stealing market share from Nvidia at the moment. Groq and Tenstorrent are extremely promising, but both are still private companies. Once Groq goes public, Nvidia will tank a bit for a while while all the "experts" announce the end of Nvidia. I wouldn't be surprised if then Nvidia would then also sell specialized AI accelerators, if they find that segment attractive enough due to losses in general GPU demand created by those companies.


To be pedantic, AMD have dramatically grown their data center share with their alternatives over the past quarter. So there is definitely some market share being lost.


What has it been delivering? Who’s making money from this stuff?

To quote Steve Jobs when talking to Dropbox: you guys don’t have a product, you have a feature.


(Shrug) Jobs is dead, and Dropbox has an $8B market cap.


And what's Apple's services revenue off of iCloud?


I'm AI-positive (now), but yes this sounds like a chip bubble. NVIDIA seem to be good at chasing these bubbles -- first crypto mining, now AI. It wouldn't surprise me to find one of the major buyers is a speculator (hedge fund led by crypto bros, for example).


I'm crypto bearish and AI-neutral but it seems less to me like NVIDIA chasing bubbles and more like new and interesting applications for the type of compute that NVIDIA offers keep emerging.


I've seen gluts not followed by shortages, but I've never seen a shortage not followed by a glut.

- Nassim Nicholas Taleb, Twitter, 2021-09-11

https://x.com/nntaleb/status/1436776641536090117


From what I remember public companies have to disclose any customer responsible for more than 10%+ of their revenue on their 10-K so those won't be "mystery whales" for long.


Surely an intelligence agency or two would be big buyers. They could lease but that might have security implications they're not comfortable with.


I’m kind of shocked that we’re not seeing any new consumer GPU products from them. It’s like they’re content to just give that market away to AMD.


Rumor is that AMDs RDNA4 will only span the low-to-mid range with no new flagship until RDNA5, so if anything they are the ones ceding the (high end) market to Nvidia.


Nvidia's grasp of desktop GPU market balloons to 88% — AMD has just 12%, Intel negligible, says JPR https://www.tomshardware.com/pc-components/gpus/nvidias-gras...


Nvidia is on a two-year schedule and the 5090 should launch later this year.


What about more affordable cards? Early 2025?


Looking at the release dates from RTX 4000 [0] it's likely, yes

[0] https://en.wikipedia.org/wiki/GeForce_40_series


Standard rollout process for the rest of the year as like the last generations


Define affordable. What minimum specs and what price point?


Honestly, I predict none of the cards will be affordable.


Shocked? It's pretty clear they dominate the market, they set the price/perf and release windows. AMD gave up being a disruptor and just follows them.


It'd be crazy of one of them is RenTech


It will be interesting to see how AI opportunities evolve and if open source models will play the same role as the public infrastructure of the dotcom boom did.

Or if closed models will dominate. For example, by the largest companies leveraging their existing distribution channels and/or acquiring promising startups.


I have a few of my customers using AI and they are asking me to build self owned AI server running open source models. With about $20k you can have your own little AI beast and do a lot with it.

They do this because proprietary AI models are not flexible enough and are lacking a lot of API.

For example, one app I wrote was to analyze scans of old maps and use generative AI to extrapolate and create animations.

I don't know where the market will go. But my feeling is that large proprietary models are very good at a very limited type of work and that open source will provide diversity.


That’s interesting. Are companies coming to you and you propose the technical solution for their problem statement?


Yeah, I build custom solutions. For one customer I built a few custom server using supermicro chassis.


> Nvidia's net income margins are staggeringly high, with $5.60 out of every $10 of revenue

Competition when ? Are amd, Intel or other companies in the situation to be able to eat some of nvidia insane margin ?


Going after this exact market is potentially a fool's errand.

AMD and Intel would probably be better off researching entirely different approaches that they can leverage their existing expertise for - i.e., some architecture that relies heavily on efficient OoO processing pipelines and free (if predicted correctly) control flow changes. Techniques that are antagonistic to GPU processing could represent a competitive moat.

Joining an existing rabbit chase right in the middle can quickly evolve into a catastrophic strategic choice when the cost of entry is billions of dollars.


Your comment relies on the theory that a.i. is a bubble.

What if companies still keep buying insane amount of graphic card for the next 20 years ? At some point, other companies will want to eat some of the cake too.

What about Chinese? They always want to have Chinese made hardware because of the fear of spying and trade war, they too have extreme need for graphical power and nvidia is a Delaware company.

It seems there are multiple reason for competitors to step up.


What happens when the new models come out and the data centers are full of old models to be decommissioned. Would love to buy a huge amount of h200 once they have become “obsolete”


The consolidation of progress into the hands of a few should be fought against at (almost any) cost. Run Linux!!


But the how will I run MS Teams, office365 and OneDrive?

(I’m only partially kidding, sigh…)


If you’re serious, they all have web apps. I use them on my linux box all the time. Ripped the certs off my corp. laptop.


Teams: several options

Office 365: several options

OneDrive: several options

Check out https://github.com/awesome-selfhosted/awesome-selfhosted


95% of corporate environments: 1-2 options. Windows or macOS for the chosen few.

At least the overloads I have worked for don’t allow us access unless the machine is locked down so hard that it’s borderline unusable.


I’m past the edit time, but “unless” should be “and”


Word on the street is that port 53 tends to stay open. YMMV.


The problem is that we typically can’t install software. Also subverting the corporate firewall can result in a rapid loss of income.


low quality discussion, lots of ignorant speculation... I wonder if there's a way to analyze HN discussions to measure "quality" ?


One indicator that I find reliable is simply if the comments exceed the up votes.


Apparently HN downranks posts using the same indicator.


Would it be legal for Jensen to be one of them?


Fun note, you can structure options and “RSUs” with allocations of products, RSU in quotes because the S stands for stock and you wont be giving shares

One benefit of non-securities underlying assets is that you can play with their pricing a lot more. like, you can have your friends vesting on some shoes you control the issuance of - or GPUs in this case - at a 99% discount and there’s no reporting or regulation to a government over this. Big problem to do that with shares.


There's zero evidence of this. He's cashing out $1B per year but round-tripping that money back into Nvidia would just cause problems for him.


I'm curious, why would he be one of them?


NSA are surely one of them.


TLAs procure indirectly for obfuscation and regulatory reasons. They wouldn't directly make purchases at this size.


They were one of the first HDTV customers, along with NGIA and NRO.


Don't they outsource work to places like Palantir? Although I can easily imagine bosses of these 3 letter agencies scrambling over each other in another glorious fit of FOMO in their internal race of 'who can model every single human on earth better'


The amount of ignorance surrounding that agency on this forums is truly astounding.


Hoid it through the grapevine


Yeah planning to get out of NVIDIA shares after Blackwell.

There is also an alarming (as shareholder) rise in custom silicon. Groq sambanova cerebus etc.


I mean. It’s no mystery is it. Microsoft. meta, Amazon and even Oracle


Let's play "Guess the whales!"

N _ _

_ B _

_ _ A

_ H _


Very sustainable, very cool. Awesome market economy we have guys.


Those whales - probably major cloud vendors - likely have the resources to develop their own hardware at some point.

Right now it’s “buy GPUs at any cost”. If things slow, there will be a chance for these customers to consider how to optimize this cost. NVIDIA can’t sit on its laurels like Intel did with x86.


Google did this a decade ago. Amazon has a CPU but I think no GPU yet although Im sure its being worked on. The problem for them is the CUDA moat. Their hardware is mostly used for inference because no one trains on non-nvidia hardware.


I don't really understand how CUDA is proper moat... You can scale software engineers much more readily than hardware supply chain. And basically for your own models, it should not be impossible to train your staff to use your layer instead.


>it should not be impossible to train your staff to use your layer instead.

First, it's the classic chicken and egg problem. Why would you invest in a CUDA alternative when you're going to be using nvidia hardware anyway?

Second, something can be not impossible but still quite difficult. As AMD and Intel have shown, creating a GPGPU API for your hardware that people want to use is not a trivial task and to date have not managed to do it.

Lastly this must just be differences in our experiences with cooperate management, because mine has been that in general they would always prefer to spend on stuff over headcount if said stuff reduces the headcount required.


Im inclined to agree, I dont think it's a sustainable moat. But AMD and intel have been trying to break through and have had minimal success so far. Until someone actually releases a CUDA competitor that is used the moat exists.


Critical thing here in my mind it is not so much general moat, but lot of individual moats. Each of these companies investing billions can build their own. Easily.

So most probable end result is that we end up with multiple competing alternatives all with their own vendor lock ins. And general public might be lucky to get one or two options.


>You can scale software engineers much more

People do not scale.


> The problem for them is the CUDA moat.

Compared to the problem of developing a new cutting-edge GPU, building a CUDA compatibility layer is a much smaller problem. Hire the author of ZLUDA, throw a small team at it, and have a legal department on standby. And separately, there'd also be value in some source-translation projects to help people migrate to some better native framework.


Both in terms of hardware and software, they need to be able to support the small set of operations that their training or inference uses, so the problem on both sides is much smaller than both a full GPU and a full CUDA replacement.


Why support proprietary libraries? Isn't it better to make an open-source library.

Also as for AMD, as I understand, they are unwilling to make ML libraries for consumer-grade GPUs and GPUs built into CPUs.


An Open Source compatibility layer for CUDA to allow people to run on non-NVIDIA GPUs is a first step; it removes the lock-in between GPU and library. Once people can run all their existing software on a different GPU, they can then consider adopting a better standard to build on top of.

The reverse approach, of trying to entice people to move from CUDA to a different library and switch GPUs at the same time, has been tried repeatedly and has not yet succeeded. Trying something different seems warranted.



What if having the resources to develop hardware are not the point? This is a physical business, and supply chain is the bottleneck at some point. Right now it seems that all the money in the world can't build fabs fast enough to manufacture alternatives to Nvidia's chips. As long as they maintain dominance over the supply chain, having developed equivalent technology might not matter. Someone correct me if this is wrong, I'm mostly speculating.


I mean it's all TMSC at the end of the day. They have finite capacity that's all tied up for a while.


The future order book is the interesting part. Does Nvidia has as big share as they do now or has someone else paid more that is outbid them.


> Although the names of the mystery AI whales are not known, they are likely to include Amazon, Meta, Microsoft, Alphabet, OpenAI, or Tesla.

I hope they're not saving their best chips for the likes of Tesla/Grok. That'd be a PR nightmare if and when it leaks.


Given their near monopoly on the products they're selling, anti-competitive behavior like that would actually be pretty big liability.


Not it would not. Nvidia has never been the good guy in the eyes of the public and most people buy Nvidia because they are better than the competition. Getting in bed with Elon would just be seen as a capitalist company doing capitalist things.


Yeah, I don't really see what you'd buy in place of Nvidia? Either you're huge and have the funds to do your own chips, or you're stuck buying Nvidia, or maybe you do both.


Yeah, can you imagine a business selling to the bad space man. Don’t they know they’re supposed to hate him?


Well, he is known for stiffing his contractors and creditors.


No, he isn't.



Whether he did or not is not the same as saying he is "known for it".

If you asked 1000 random people to say what they know about Elon musk, what percent do you think will say "oh. You mean the guy that doesn't pay vendors!"

Also. What is the base rate for contract disputes with vendors among large companies? He runs 3 large companies, surely with tens of thousands of contracts for services. There will always be disputes there - is his rate higher than average? Does he lose very dispute in court?


There are plenty of articles calling his companies out specifically and few calling other similar companies out so you can take your what ifs somewhere else.


Maybe because anything "Elon musk", including what he randomly tweets on the toilet, is news, but a random contract dispute with GE and a vendor is not news.


He is. Read the web (and the room) friend.


"Read the room" is just a variant of the appeal to authority fallacy.


He is.


[flagged]


If you post irrelevant or thoughtless takes, you will get vote down by anyone, even those who don’t like musk.


You're entitled to your opinion, but I always strive for thoughtfulness and relevance.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: