Only CUDA/Nvidia managed to build something profitable with it, a competitive advantage that could also go down the drain once more investment is put into CUDA alternatives.

And this is counting on AI hype to continue (likely, but maybe demand for hardware might eventually stabilise).

Even though semiconductors is a highly specialized industry, it's a CAPEX heavy business, you need both fabs and research and you also have too many players in the market.

People talk a lot about TSMC and the bleeding edge business(computers, phones), but that's just one part of it, but there's a whole industry of embarked software that will likely be much bigger.

I don't doubt that the sector will grow(in revenues), but I doubt they'll be able to produce a profit as high as some companies in the tech sector like advertising etc, that are much more asset-light.

I really fail to see what people see even in Nvidia/Cuda, it won't take long until others catch up, and they'll likely move back to margins closer to the industry, which are extremely low. There's even more money to be made in sectors like energy drinks.

It's a high-margin business!

I suspect that to some degree individual drinks will be boom -> bust, as newer brands cannibalize the market by having (or advertising) superior energy, more caffeine, better comedowns, etc., and with older drinks having a higher likelihood of bad publicity simply on account of having been around longer. MLMs like Herbalife are also operating in the space (all "loaded tea" shops are MLM), so obviously in cases like that there's a ton of pressure producing early growth.

Is there that much more potential left for energy drinks?

I would hope not for the sake of people’s health.

President Camacho has yet to come to power to authorize energy drinks for irrigation (because it's got electrolytes), so I'd say yes, there's still a ton of potential.

Coca Cola and Pepsi each seem to have profits in the $5B to $10B range, which can be a proxy for how much profit there exists in the space because everyone else is much smaller.

Maybe if these energy drinks displace all of coffee and tea, but I would be surprised. How much more caffeine will (or should) consume?

TSMC literally can't make chips fast enough. Companies are competing for wafers.

>you also have too many players in the market

How? There are a handful of companies on the planet that can do 7nm process chips and below. Western Digital had an issue with its fab in 2022 and that alone was apparently enough to increase NAND flash prices by 10%.

https://en.wikipedia.org/wiki/Beer_distribution_game

At 2020, a lot of people were saying that no, the executives on those companies know about the problem, of course they'll be able to avoid it. Yet, here we are. (Anyway, the game is supposed to be played by people that were instructed about the problem. But you can try it with naive people before instructing them too. It doesn't make a difference.)

TSMC specifically may be able to avoid it. They are in a different market from most players, with less players that maybe are close enough to act to avoid it. I still wouldn't bet on it.

That's interesting since Samsung is their primary competitor at the advanced nodes. It takes time and money to move though as fabs aren't completely fungible.

Your comment >> TSMC literally has an entire paragraph in parents comment that addresses it.

That looks like a single sentence, not an entire paragraph... And it's a paragraph without supporting evidence. No shocker c7DJTLrn wasn't convinced about parent commenters weak claim.

The paragraph is so obvious that I would never expect anyone to question it. TSMC is a mere 12.5% of global semiconductor market in 2022.

There is, however, "consumer chips foundry market", which is dominated by TSMC with 55%+ market share. And TSMC's market share is limited only because of their production capacity.

It is market of small series custom devices, prototypes (like Oculus Rift), aftermarket upgrades, like cars tuning.

Yes, creators market is not too large, total number of sells just around million, but it exists.

BTW it is interest question, how large part of steel market also belong to creators market, if it is larger than semiconductor creators market.

>People talk a lot about TSMC and the bleeding edge business(computers, phones), but that's just one part of it, but there's a whole industry of embarked software that will likely be much bigger.

I have no idea what "embarked software" means. So-called "bleeding edge" makes up a significant portion of semiconductor profits, so how can you disregard that and say semiconductor is going bust?

OP's name sounds brazillian, where embedded is called embarcado.

Those semiconductor companies can make lots of revenue, but operate on thin margins, because a competitor can also build a fab. Think graphic cards, Radeon(now AMD) has consistently built worse graphic cards, with less R&D, but sold at a cheaper price point and survived throughout the years.

TSMC has some extra expertise that others doesn't, and has good profit margins because of that, but together with NVidia, they are the champions, and others are constantly catching up. As with many things in a capitalist system, prices face a race to the bottom.

Related to bleeding edge fab. 14nm has suffered some shortages, but even that has changed.

One example is Apple, even though they've managed to increase their revenues (and products sold) a lot because of Covid, that demand is down. Apple is projected to have less revenues this year than the last years.

The same has happened with PC sales. What has been really driving revenues and profits up in this space is AI.

I don't believe this will continue. More competition will drive profits down, meaning it'll be a less attractive field to invest for VCs or investors.

Semiconductor business is more akin to package food products in my point of view. Some products can enjoy a nice margin because of their brand, taste or being hyped at a specific moment, but the majority of it compete on thin margins.

It isn't like Meta, a closed garden, that has plenty of data and have a product that got everybody addicted and dependant on it, either social networks or communications, that because of network effects is extremely sticky. Where only regulations and government intervention could pose a risk to it.

But wall street and people's expectations(and valuations) on semiconductors is suddenly as if they are building the next monopoly. Monopolies only exist because there's only one, what I see is a reaaaally fragmented market, with high CAPEX, thin margins and low moat.

Building new fabs, especially for the advanced nodes, costs billions, takes years, and requires a lot of know how and special equipment. There are only a handful of companies operating and building new fabs.

> TSMC has some extra expertise that others doesn't [...] but together with NVidia, they are the champions, and others are constantly catching up

Not sure what you mean. Nvidia uses TSMC for the manufacturing for the 40 series (and Samsung for the 3000 series) and doesn't have their own fabs.

> Think graphic cards, Radeon(now AMD) has consistently built worse graphic cards, with less R&D, but sold at a cheaper price point and survived throughout the years.

You mean ATI (now AMD)? They got rid of their fabs back in 2009 and put them into GlobalFoundries. They now also mostly use TSMC for their CPUs and GPUs (and profit from the more advances nodes compared to GF) and GF for a few things in larger nodes.

Apple also (just as everyone else you have mentioned) uses TSMC for (most of) their chips and is one of TSMCs largest customers. Intel and Samsung have their own fabs, but also seem to lag behind TSMC.

I'm also sure here people know how fabs are expensive.

I believe you understood my comment related to Radeon being part of AMD. And also you need to work on being less pedantic.

Cloud providers are choosing nvidia today as the gpgpu platform on the cloud, driven by neural network needs. So lots of software will be written with the platform in mind, aka. for nvidia arch / in cuda.

This will add friction preventing users to easily change platform (see how long it took for people to switch their workloads to arm for example). It will drives more cloud providers to use nvidia, more software to be written for cuda, and the moat will build itself.

That's true, but the current bottleneck in AI/ML is memory size+ bandwidth. NVidia with their cuda really is the most economic game in town right now on both the Low "hobbyist level" end (nothing beats 2x used rtx3090 setup in value for money), up to "high end" (h100 is still the best generally available card). Some interesting "unified memory" competition appears on the horizon(Apple, AMD), but both are still far from where nvidia is.

If somebody would come up with a product Tomorrow that isn't as performant as H100s, but price-wise is cheaper, they could make NVidia miserable, as NVidia is selling those products with a huge margin, they'd have a hard time keeping those margins and a price race to the bottom would happen.

If you get around that, sometimes projects will use dependencies that include an odd CUDA-only library. I ran into this a lot with GANs (which often use some weird loss metric with a random hard CUDA dependency) or media libs.

Of course there are plenty of solutions, and some are theoretically transparent. But the issue is getting people to use them. Academic ML researchers in particular seem incredibly impatient, looking for the absolute easiest way to prototype something (which usually means coding what they already know). Ease of installation on a wide range of HW is about their last priority.

The kernels I wrote I hand-optimized using Nsight for a specific usecase, and sure they saved me like 200% compared to pure pyTorch and it took a few days, but when your company spends $10M for a single training run, you have quite many of "my days" to spend if another HW platform would save you $5M (but you need to have the calendar time as well to spend, which is another factor in this space that moves crazily fast).

If/when the field stabilizes a bit people will start looking at that bottom line I think but right now everybody just "has" to get those H100 cards before someone else gets them I guess...

There are a lot of optimizations that CUDA has that are nowhere near supported in other software or even hardware. Custom cuda kernels also aren't as rare as one might think, they will often just be hidden unless you're looking at libraries. Our more well known example is going to be StyleGAN[0] but it isn't uncommon to see elsewhere, even in research code. Swin even has a cuda kernel[1]. Or find torch here[1] (which github reports that 4% of the code is cuda (and 42% C++ and 2% C)). These things are everywhere. I don't think pytorch and tensorflow could ever be agnostic, there will always be a difference just because you have to spend resources differently (developing kernels is time resource). We can draw evidence by looking at Intel MKL, which is still better than open source libraries and has been so for a long time.

I really do want AMD to compete in this space. I'd even love a third player like Intel. We really do need competition here, but it would be naive to think that there's going to be a quick catchup here. AMD has a lot of work to do and posting a few bounties and starting a company (idk, called "micro grad"?) isn't going to solve the problem anytime soon.

And fwiw, I'm willing to bet that most AI companies would rather run in house servers than from cloud service providers. The truth is that right now just publishing is extremely correlated to compute infrastructure (doesn't need to be but with all the noise we've just said "fuck the poor" because rejecting is easy) and anyone building products has costly infrastructure.

[0] https://github.com/NVlabs/stylegan2-ada-pytorch/blob/d72cc7d...

[1] https://github.com/microsoft/Swin-Transformer/blob/2cb103f2d...

[2] https://github.com/pytorch/pytorch/tree/main/aten/src

Semiconductors are cyclical in general, and most sources that I trust believe that the bottom is already behind us. Many of the big players have already begun guiding revenue growth again, though not as massive as Nvidia's windwall, obviously.

I don't mean to imply your life is over if you graduate during a downturn but you can easily end up derailed and with a potential hit to your earnings going forward.

(This is also because of where I graduated, and the fact that I didn't feel comfortable moving to the US post-9-11)

Of everyone, you'd think Microsoft would have the most incentive (and ability) to improve tooling/stability on not-Nvidia.

> I really fail to see what people see even in Nvidia/Cuda, it won't take long until others catch up, and they'll likely move back to margins closer to the industry, which are extremely low.

Looks like analyst predictions are that Nvidia growth will start flattening in Q1 2025.

And when even Cathy Woods is selling because of over-valuation concerns, everyone should be looking for the exits.

Part of Nvidia's strategic problem is that they made too much money recently: they needed lower profits in their primary markets to disincentivize competitor investment and retain their moat.

Instead, AMD and Intel saw a cash cow. And those aren't small, ignorable competitors.

To me it just seems like she happens to pick some stocks based on gut feeling because of some random detail she liked about it and then will say whatever to justify her positions afterwards.

The problem was, her ETFs are not paid and incentivized by performance. She’s paid and incentivized by AUM. So her team spends their time writing speeches and on Twitter writing hype posts instead of shutting up, being heads down buried in a laptop analyzing stocks, and performing analysis quietly. She might’ve been worried her AUM could drop if she made big changes like liquidating to cash.

AMD and Intel have been trying hard to compete with Nvidia for a while now. I mean in AI accelerator market. Little success so far.

If AMD had been really interested (from a corporate strategic level) in the GPGPU market, their driver/library team would be a lot bigger.

You provide a BLAS implementation, and your primary customer can take it from there.

As opposed to Nvidia's current market, which spans HPC to AWS/Azure/GCP to corporate datacenter to "I followed the directions on Reddit" personal machines.

I thought that would happen by now too, in 2016, so I started slowly selling much of my NVDA stake after a few years. This was wrong. It turns out they have created an ecosystem effect where everyone makes their papers and code work in CUDA and everything else is best effort. AMD hasn’t even really been seriously trying from my point of view, but once they do they’re still trying to bail water with a leaky bucket - new models come out every day where the researchers have run and tested on CUDA only.

Say they do catch up with the hardware, are they also going to convince the entire industry to switch from CUDA.

One of the big-enough generative/LLM AI users would be my guess. OpenAI for example seems to be even going towards making it's own chips as an alternative to Nvidia things being too expensive.

> are they also going to convince the entire industry to switch from CUDA.

Statistically unlikely, for example ARM definitely disrupted MIPS in several embedded networking niches uses, but didn't convince the entire industry to switch from MIPS.

Designing a specialised ASIC is much easier and cheaper than a general-purpose, public use processor for a wide range of applications. You don't need most of the programmability or even half the compute and fancy memory and scheduling units. You also don't need the to develop a rich API (like CUDA or Vulkan) for third parties to use.

This holds even for state of the art intensive calculation engines with minimised energy consumption. Think of all the crypto-mining ASICs that were built a few years ago. They were relatively cheap to design and optimised for calculations per energy unit.

I find it very likely that we'll have in a year or two, not only cheaper but also faster hardware that can do training and inference.

They're a nice middle ground between a GP CPU and ASICs that is well suited to the kinds of workloads that venture capitalists are, for the moment, interested in investing in. And it's possible that there are enough applications that benefit from GPU architecture that it's more efficient to force a GPU to pretend to be a CPU instead of having a CPU.

Most likely it's just another cycle of the RISC vs CISC pendulum. Like cloud vs desktop, physical vs virtual, serial vs parallel, monolithic vs micro, etc. We'll stick with CUDA and use it to solve as many problems as we can with that method, and then as we discover new ways to create or organize GP CPUs, we'll shift back to those until we find that problems get hard to solve again.

What in embarked? Genuinely curious if this is a misspelling of embedded or if it's something else entirely.

AI has a lot of similarities to the crypto bubble, but people keep repeating this "ah, but now it is useful!", sure it is. I like ChatGPT, but will people really use all those new AI products being built?

Most things in the real world need an accuracy beyond what AI can offer and will likely offer in a few decades ahead, go learn about AI and any expert will show you that for example, LLMs can't solve some basic problems.

I'm not saying AI will never be as smart as human, but rather that, it will take longer until we've figured out stuff that people are already pricing in into markets as if it will happen in a year.

The only time that’s true is if your revenues are flat year over year.

That's only true in an environment that values growth dramatically over profit.

With non-negative real interest rates - profits actually matter.

But relative changes in revenue are much more impactful than relative changes in profit. This one is obvious, because revenues are much larger and bounded at a minimum of 0.

The only time the GP is false is when companies are reinvesting profits in a way that makes it look like costs. (GP said profits are more important)

But relative changes in revenue are much more impactful than relative changes in profit (you said revenue is more important)

No. What I said means that relative change isn't a good metric for any of them. But it's much less bad for revenue than for profit.

That also makes a lot of assumptions on its own. Baseline is: profits matter. Capitalists want their cut. Profits is the extraction of the "thing". If capitalists can't put the "thing" in their pockets then why even bother?

So maybe the company has $100 in profit in 2024, and they have 5 shares, and they pay out all of their profit as $20 dividend per share. And in 2025 you expect them to make $200 in profit, with the same 5 shares, which is $40 per share.

Although that's a theoretical model, and there's many companies that don't pay dividends at all and still command mighty stock prices (e.g. TSLA) this is how real companies are actually valued -- on the expectation of future cash flows, maybe times some multiple.

The reason why revenue is more important than profits for the vast majority of technology / growth-stage companies is that most of these companies are focused on creating double-digit percent revenue growth every single year. Their profits will be 0 or negative because all of the money that they make from selling widgets is getting spent on marketing or hiring new engineers to build more widgets, or building a factory.

But after 5-10 years, this company will have "scaled over" its fixed expenses like rent, they will have huge revenues, they'll have serious market share, and then you can simply slow down / turn off the marketing and other SG&A expenses, and then suddenly you go from being a 0 or negative profit company to a multi-billion $ profit company.

But if you're too focused on clipping coupons and scrounging pennies as a startup founder, you aren't focused on the right aspect: revenue growth.

Gotta look at stock buybacks too, which are effectively the same thing but more tax efficient for investors. (Though TSLA doesn't do significant amounts of that, and many other giant companies also don't do it relative to profits either, so your general point still stands)

The theory is useful to understand that there's some basis to the share value of equities although it probably doesn't have a lot of practical application.

Is this some kind of reporting error, or is that number accurate?

from TFA

Looks like, AMD was right, to separate fabs and become fabless, and SEC for some reason cannot do it and suffer losses.