Buy a used workstation with 512GB of DDR4 RAM. It will probably cost like $1-1.5k, and be able to run a Q4 version of the full deepseek 671B models. I have a similar setup with dual-socket 18 core Xeons (and 768GB of RAM, so it cost about $2k), and can get about 1.5 tokens/sec on those models. Being able to see the full thinking trace on the R1 models is awesome compared to the OpenAI models.
I use a dual-socket 18-core (so 36 total) xeon with 768GB of DDR4, and get about 1.5-2 tokens/sec with a 4-bit quantized version of the full deepseek models. It really is wild to be able to run a model like that at home.
Yeah, it was just a giant HP workstation - I currently have 3 graphics cards in it (but only 40GB total of VRAM, so not very useful for deepseek models).
Interns and new grads have always been a net-negative productivity-wise in my experience, it's just that eventually (after a small number of months/years) they turn into extremely productive more-senior employees. And interns and new grads can use AI too. This feels like asking "Why hire junior programmers now that we have compilers? We don't need people to write boring assembly anymore." If AI was genuinely a big productivity enhancer, we would just convert that into more software/features/optimizations/etc, just like people have been doing with productivity improvements in computers and software for the last 75 years.
Isn't that every new employee? The first few months you are not expected to be firing on all cylinders as you catch up and adjust to company norms
An intern is much more valuable than AI in the sense that everyone makes micro decisions that contribute to the business. An Intern can remember what they heard in a meeting a month ago or some important water-cooler conversation and incorporate that in their work. AI cannot do that
AI/ML and Offshoring/GCCs are both side effects of the fact that American new grad salaries in tech are now in the $110-140k range.
At $70-80k the math for a new grad works out, but not at almost double that.
Also, going remote first during COVID for extended periods proved that operations can work in a remote first manner, so at that point the argument was made that you can hire top talent at American new grad salaries abroad, and plenty of employees on visas were given the option to take a pay cut and "remigrate" to help start a GCC in their home country or get fired and try to find a job in 60 days around early-mid 2020.
The skills aspect also played a role to a certain extent - by the late 2010s it was getting hard to find new grads who actually understood systems internals and OS/architecture concepts, so a lot of jobs adjacent to those ended up moving abroad to Israel, India, and Eastern Europe where universities still treat CS as engineering instead of an applied math disciple - I don't care if you can prove Dixon's factorization method using induction if you can't tell me how threading works or the rings in the Linux kernel.
The Japan example mentioned above only works because Japanese salaries in Japan have remained extremely low and Japanese is not an extremely mainstream language (making it harder for Japanese firms to offshore en masse - though they have done so in plenty of industries where they used to hold a lead like Battery Chemistry).
> by the late 2010s it was getting hard to find new grads who actually understood systems internals and OS/architecture concepts, so a lot of jobs adjacent to those ended up moving abroad to Israel, India, and Eastern Europe where universities still treat CS as engineering instead of an applied math disciple
That doesn’t fit my experience at all. The applied math vs engineering continuum is mostly dependent on whether a CS program at a given school came out of the engineering department or the math apartment. I haven’t noticed any shift on that spectrum coming from CS departments except that people are more likely to start out programming in higher level languages where they are more insulated from the hardware.
That’s the same across countries though. I certainly haven’t noticed that Indian or Eastern European CS grads have a better understanding of the OS or the underlying hardware.
> I certainly haven’t noticed that Indian or Eastern European CS grads have a better understanding of the OS or the underlying hardware.
Absolutely, but that's if they are exposed to these concepts, and that's become less the case beyond maybe a single OS class.
> except that people are more likely to start out programming in higher level languages where they are more insulated from the hardware
I feel that's part of the issue, but also, CS programs in the US are increasingly making computer architecture an optional class. And network specific classes have always been optional.
---------
Mind you, I am biased towards Cybersecurity, DevOps, DBs, and HPC because that is the industry I've worked on for over a decade now, and it legitimately has become difficult hiring new grads in the US with a "NAND-to-Tetris" mindset because curriculums have moved away from that aside from a couple top programs.
ABET still requires computer architecture and organization. And they also require coverage of networking. There are 130 ABET accredited programs in the US and a ton more programs that use it as an aspirational guide.
Based on your domain, I think a big part of what you’re seeing is that over the last 15 years there was a big shift in CS students away from people who are interested in computers towards people who want to make money.
The easiest way to make big bucks is in web development, so that’s where most graduates go. They think of DBA, devops, and cybersecurity as low status. The “low status” of those jobs becomes a bit of a self fulfilling prophecy. Few people in the US want to train for them or apply to them.
I also think that the average foreign worker doing these jobs isn’t equivalent to a new grad in the US. The majority have graduate degrees and work experience.
You could hire a 30 year old US employee with a graduate degree and work experience too for your entry level job. It would just cost a lot more.
So applying this to China and the USA - the USA has a median household income of ~$80k, and China, in terms of purchasing power parity, has a median household income of $32k. China has a workforce of ~775M people vs 163M people in the USA. The USA has ~7M unemployed people. For the USA to stop importing goods from China, we would need to 1) stop consuming those goods altogether (even if they're intermediate goods that go into things we do manufacture), or 2) employ US workers to make those goods instead of whatever they're currently doing (either employed or unemployed).
Employed Americans largely earn a lot more than employed chinese people, and there just aren't very many unemployed Americans. Based on the value of imports to the US from China vs Chinese GDP($440B vs $17.8T), we import about 2.5% of their output, or the equivalent of ~20M people's output if we naively scale. I don't think there is any way around the fact that significantly reducing imports from other countries means we significantly reduce our consumption in absolute terms (i.e. we have a poorer standard of living), just out of spite for other countries.
> the equivalent of ~20M people's output if we naively scale
Or about 12% of the US labor force. Meanwhile the 12th percentile US income is ~$22,000, i.e. less than the $32k median in China, and half of the jobs there are above that median.
Obviously these numbers are all useless napkin math and none of it really works like that, but the premise isn't inherently absurd. If China is subsidizing manufacturing in order to capture manufacturing jobs and those jobs pay more than what many in the lowest quartile of the US are currently making, there are people who could be made better off by having those jobs back, and they'd still only be paid about what we're currently paying to China.
And this before considering the general arguments about advantages to proximity of manufacturing, e.g. being able to talk to the people in the factory in your timezone in your language assists in product development so the location of the factory has an easier time developing new products. Which allows not just the manufacturing jobs but the whole rest of the company to be there, instead of those jobs starting to get eroded going forward.
The US manufactures more than ever, and the % of GDP in manufacturing has been flat for 70+ years now. The jobs didn’t leave, so they’re never coming back. They got automated. Just like we need vastly less farmers than in 1900 to farm more than ever, we need vastly manufacturing jobs to manufacture more than ever.
More of what, exactly? I looked at steel production data (in metric tonnes, not dollar value), US production has been flat or slightly declining since 1970.
People normally show a slowly increasing graph to claim that the US manufactures more than ever- but it's primarily because of hedonic adjustments for Intel chips. If you take that out it falls off a cliff in the 2000s during the China Shock.
Computer components, medical equipment, vehicles, aircraft, chemicals, etc.
Steel is something where value matters. US steel is usually higher grade or recycled with more value. Korea churns out tons and tons of low value low margin rebar.
You bring back manufacturing by investing in clustered industries. New York has a bunch of semiconductors. There’s a ton of chemical industries around Philly and Houston.
The iPhone is made in China because they have the best manufacturing for precision products. If you want that here, you have to invest.
Automation increases production efficiency. Jevon's paradox says this will generally increase consumption, and in fact it has, so US consumption has gone up but production is flat. The difference comes from increasing imports from manufacturing in China. If the US manufactured the additional stuff instead of importing it, it would have more manufacturing jobs than it does now.
This is why the US needs automation. You don't want jobs assembling iPhones by hand, you want jobs building and maintaining factory automation equipment. But this is the trend globally anyway because a) it's becoming increasingly possible and b) the wages in countries like China have been increasing, so there is less "cheap labor" available in the alternative. But if you have an automated factory that requires trained workers making middle class money instead of a sweatshop, then why would the US want it in China instead of at home?
Decent paying manufacturing jobs are never returning; they’re automated out now.
It’s like wishing all of America would return to farming jobs, which once employed the majority of people. Then farming got automated such that now only a few percent of workers produce vastly more than those ancient hordes.
That’s where manufacturing went, and the rest of the world is as surely automating out those jobs too.
Back of the napkin, we could spend 3% of the federal budget to pay each of those 20 million people an extra $10k a year, and avoid destroying some 10% of the global economy. If we wanted to do that, I guess.
No one will hire the people you are describing. We had 3.5% unemployment, before Trump began destroying the world economy. People in most fast food restaurants are making $30k. In fact the “entry level workers” and “illegals” were relatively happy. It’s the middle class workers who vary between desperate and enraged, because $45-$65k does not approximate the expectations raised in their childhood. What I’m saying is vastly oversimplified, but you get my drift. There is tremendous housing shortage, due to government policies that surreptitiously favored rising prices for incumbent homeowners, and corporate speculators. Healthcare is apportioned according to education level. Food is remarkably expensive.
Maybe where you live people making under $30K for even the lowest entry level job is exceedingly rare, but there are still ass tons of people earning below that, especially in rural areas. Half of US states have a median wage under $40K, that leaves a LOT of room for jobs paying below $30K. And no those people are not happy, yeah housing is cheaper, but goods and services are not. Amazon doesn't charge you less for living in Mississippi instead of New York. Many stores that do still exist farther out have higher shelf markups too except for the highest demand bulk products where competition actually exists.
Ha! When I was first learning to program in high school, I wrote a 'distributed monkeys-on-typewriters' simulator. I somehow acquired a stack of surplus Pentium 100s that I had running in an unused closet at the school, communicating with each other over IPX. I remember the server had a fun 'Guess-operations-per-second' (GOPS) realtime display.
None of that means that the current companies will be profitable or that their valuations are anywhere close to justified though. The future could easily be "Open-weight models are moderately useful for some niches, no-name cloud providers charge slightly higher than the cost of electricity to use them at low profit margins".
Dot-com boom/bubble all over again. A whole bunch of the current leaders will go bust. A new generation of companies will take over, actually focused on specific customer problems and growing out of profitable niches.
The technology is useful, for some people, in some situations. It will get more useful for more people in more situations as it improves.
Current valuations are too high (Gartner hype cycle), after they collapse valuations will be too low (again, hype cycle), then it'll settle down and the real work happens.
The existing tech giants will just hoover up all the niche LLM shops once the valuations deflate somewhat.
There's almost a negligible chance any one of these shops stays truly independent, unless propped up by a state-level actor (China/EU)
You might have some consulting/service companies that will promise to tailor big models to your specific needs, but they will be valued accordingly (nowhere near billions).
Yeah, that's probably true, the same happened after the dot-com bubble burst. From about 2005-15 if you had a vaguely promising idea and a few engineers you could get acqui-hired by a tech giant easily. The few profitable ones that refused are now middle-sized businesses doing OK (nowhere near billions).
I don't know if the survivors are going to be in consulting - there is some kind of LLM-base product capability, you could conceivably see a set of LLM-based products building companies emerge. But it'll probably be a bit different, like the mobile app boom was a bit different from the web boom.
That's been the 'endgame' of technology improvements since the industrial revolution - there are many industries that mechanized, replaced nearly their entire human workforce, and were never terribly profitable. Consider farming - in developed countries, they really did replace like 98% of the workforce with machines. For every farm that did so, so did all of their competitors, and the increased productivity caused the price of their crops to fall. Cheap food for everyone, but no windfall for farmers.
If machines can easily replace all of your workers, that means other people's machines can also replace your workers.
Yeah, the overblown hype is a feature of the hype cycle. The same was true for the web - it was going to replace retail, change the way we work and live, etc. And yes, all of that has happened, but it took 30 years and COVID to make it happen.
LLMs might lead to AGI. Eventually.
Meanwhile every company that is spruiking that, and betting their business that that's going to happen before they run out of VC funding, is going to fail.
I think it will go in the opposite direction. Very massive closed-weight models that are truly miraculous and magical. But that would be sad because of all the prompt pre-processing that will prevent you from doing much of what you'd really want to do with such an intelligent machine.
I expect it to eventually be a duopoly like android and iOS. At world scale, it might divide us in a way that politics and nationalities never did. Humans will fall into one of two AI tribes.
Except that we've seen that bigger models don't really scale in accuracy/intelligence well, just look at GPT4.5. Intelligence scales logarithmically with parameter count, the extra parameters are mostly good for baking in more knowledge so you don't need to RAG everything.
Additionally, you can use reasoning model thinking with non-reasoning models to improve output, so I wouldn't be surprised if the common pattern was routing hard queries to reasoning models to solve at a high level, then routing the solution plan to a smaller on device model for faster inference.
Exactly. If some company ever does come up with an AI that is truly miraculous and magical the very last thing they'll do is let people like you and me play with it at any price. At best, we'd get some locked down and crippled interface to heavily monitored pre-approved/censored output. My guess is that the miracle isn't going to happen.
If I'm wrong though and some digital alchemy finally manages to turn our facebook comments into a super-intelligence we'll only have a few years of an increasingly hellish dystopia before the machines do the smart thing and humanity gets what we deserve.
By the time the capital runs out, I suspect we'll be able to get open models at the level of current frontier and companies will buy a server ready to run it for internal use and reasonable pricing. It will be useful but a complete commodity.
I know folk now that are selling, basically, RAG on lammas, "in a box". Seems a bunch of mid-level at SME are ready to burn budget on hype (to me). Gotta get something deployed in the hype-cycle for quarterly bonus.
I think we can already get open-weight frontier class models today. I've run Deepseek R1 at home, and it's every bit as good as any of the ChatGPT models I can use at work.
Which companies? Google and Microsoft are only up a little over the past several years, and I doubt much of their valuation is coming from LLM hype. Most of the discussions about x.com say it's worth substantially less than some years ago.
I feel like a lot of people mean that OpenAI is burning through venture capital money. It's debatable, but it's a huge jump to go from that to thinking it's going to crash the stock market (OpenAI isn't even publicly traded).
The "Magnificent Seven" stocks (Apple, Amazon, Alphabet, Meta, Microsoft, Nvidia, and Tesla) were collectively up >60% last year and are now 30% of the entire S&P500. They are all heavily invested in AI products.
I just checked the first two, Apple and Amazon, and they're trading 28% and 23% higher than they were 3 years ago. Annualized returns from the SP 500 have been a little over 10%. Some of that comes from dividends, but Apple and Amazon give out extremely little in the way of dividends.
I'm not going to check all of the companies, but at least looking at the first two, I'm not really seeing anything out of the ordinary.
Currently, Nvidia enjoys a ton of the value capture from the LLM hype. But that's a weird state of affairs and once LLM deployments are less dependent on Nvidia hardware, the value capture will likely move to software companies. Or the LLM hype will reduce to the point that there isn't a ton of value to capture here anymore. This tech may just get commoditized.
Nvidia is trading below its historical PE from pre-AI times at this point. This is just on confirmed revenue, and its profitability keeps increasing. NVIDIA is undervalued right now
Sure, as long as it keep selling $130B worth of GPUs each year. Which is entirely predicated on the capital investment in Machine Learning attracting revenue streams that are still imaginary at this point.
> None of that means that the current companies will be profitable ... The future could easily be "Open-weight models are moderately useful for some niches, no-name cloud providers charge slightly higher than the cost of electricity to use them at low profit margins".
They just need to stay a bit ahead of the open source releases, which is basically the status quo. The leading AI firms have a lot of accumulated know-how wrt. building new models and training them, that the average "no-name cloud" vendor doesn't.
> They just need to stay a bit ahead of the open source releases, which is basically the status quo
No, OpenAI alone additionally need approximately $5B of additional cash each and every year.
I think Claude is useful. But if they charged enough money to be cashflow positive, it's not obvious enough people would think so. Let alone enough money to generate returns to their investors.
After being a professional programmer for ~20 years, and recently playing around with leetcode - my main issue with leetcode is that there's almost no overlap between leetcode problems and the problems I actually encounter in the wild. The validation tests often have silly corner cases that force you into a single answer to avoid timing out. It's frequently as much work to understand what the problem is actually asking you as it is to implement a solution. Just like I've found ChatGPT to be pretty mediocre at writing the sort of code I work on, but others swear by it, maybe some peoples' dayjob actually looks like writing leetcode all day? I know a lot of interviewers use it, but it feels so disconnected from actual engineering work.
I'm working on sort of 'graph' library. It's litcoding all the way. There are many separate containers and algorithms. The problem to a) write them b) optimize for memory c) optimize for performance d) find a 'good' balance where 'good' is undefined. But it starts with architecture which is based one some estimates of achievable functionality/performance.
My line of work (ML for medical imaging) is pretty dense with leetcodelikes, especially the classic “what’s the best time complexity? Great now whats the best space conplexity”
I married at 22 and moved to nyc. I lived through meeting peers that thought we were freaks (“you two are definitely gonna get divorced”), to the gradual normalization as we got older and more of our friends married. I knew lots of people that transitioned from young and invincible and definitely never going to get married to dismayed at coming up on 40 and feeling like they hadn’t quite started “real life” yet. I don’t like the article’s depiction of the guy “taking her under his wing” had they gotten married earlier, but I will say that there is something great about sharing your youth with someone unequivocally on your side, if life gives you the opportunity.
reply