> AGI by 2027 is strikingly plausible. GPT-2 to GPT-4 took us from ~preschooler to ~smart high-schooler abilities in 4 years.
No it's not. GPT-4 is not a "smart high-schooler". This is so hype-y, it's ridiculous.
> Over the past year, the talk of the town has shifted from $10 billion compute clusters to $100 billion clusters to trillion-dollar clusters.
Talk amongst the memesters. This is a meme.
I work in AI, and I'm building an AI animation company. But let's be realistic -- claims like these are not grounded and are going to result in distrust from stakeholders: investors, policy makers, and the general public.
Those memesters invented and deployed the current systems, are currently training the next, and raising funds for the next after.
Some scientific breakthroughs were memes at first, but some people believed in them and proved everyone else wrong. Majority were memes and are still memes. I don't think whether this is a fad or not is that interesting, what's interesting is these 'memesters' who are optimistic about these systems have all the capital, determination and brain. Unlike in other times in history where public needed some convincing, whether through President on Manhattan or with Apollo project, memesters are using their own funds so don't need to convince anyone else.
> No it's not. GPT-4 is not a "smart high-schooler". This is so hype-y, it's ridiculous.
I'm reading Feynman's lectures on electromagnetism right now, and GPT-4o can answer questions and help me with the math. I doubt that even a smart high school would be able to do it.
Just to echo what you are saying, I've read the first chapter and I thought the thesis is interesting and the writing is good but I failed to be convinced becuase it makes a lot of classic mistakes you make in science. Even though logical arguments are being made there is no attempt not to overfit the data.
The author brings out a lot of stats "smart high-schooler", "effective compute", "OOM", "Test Scores", "inference efficiency" but doesn't do a good job of explaining how the author predicted these things before hand (preregistering) and how they actual will result in new technologies or how we can extrapolate past the trend line.
Also in the unhobbling section
"Tools: Imagine if humans weren’t allowed to use calculators or computers. We’re only at the beginning here, but ChatGPT can now use a web browser, run some code, and so on. "
This is so non-specific (because no one has really commercialized anything with this yet) that I worry that we don't actually know if we can make the kinds of effective tools the author is talking about. Would love some feedback on these critisms
also one funny thing is that the author mentions power constraints, but then doesn't calculate how many terraflops for example the us grid can produce etc.
This large piece of text starts kinda convincing but then there are nuggets, like I guess glitches in the matrix, that kind-of ruin the show. for example:
"""There’s some assumptions that flow into the exact numbers, including that humans “think” at 100 tokens/minute (just a rough order of magnitude
estimate, e.g. consider your internal monologue)"""
This makes me think the whole thing is not really science. For example there is very little about energy costs and energy availability limits in there, just the amazing statement that the US only has to burn lots of its nat-gas (emitting lots of CO2 btw.) to get the power needed.
It doesn’t start off convincing though. His premise is that since 2018-2024 follows exponential growth, 2024 onward will. Contrast with Yann’s oft repeated mantra that what looks like exponential growth is more likely the beginning of a sigmoid.
That’s the difference between someone with 3+ decades in the field of AI research vs someone with barely 2 decades on earth.
This is a fair critism. However, I don't think it matters much, to the overall argument he makes.
Why?
We don't need AGI to see massive geopolitical disruption. We are already seeing this. The US has put up a GPU wall around China. That is evidence enough.
The capability we have already is enough to greatly upset the balance of power in a variety of spheres... Both sides lag in implementation. The capability is alresdy there. Capability will grow. AGI is not relevant to the concern for security. We've past that point already. The labs are a key NAT security asset. They just are privately owned/operated... For now. This is the way of things. History shows this.
>AGI by 2027 is strikingly plausible. GPT-2 to GPT-4 took us from ~preschooler to ~smart high-schooler abilities in 4 years. Tracing trendlines in compute (~0.5 orders of magnitude or OOMs/year), algorithmic efficiencies (~0.5 OOMs/year), and “unhobbling” gains (from chatbot to agent), we should expect another preschooler-to-high-schooler-sized qualitative jump by 2027.
>I make the following claim: it is strikingly plausible that by 2027, models will be able to do the work of an AI researcher/engineer. That doesn’t require believing in sci-fi; it just requires believing in straight lines on a graph.
> Tracing trendlines in compute (~0.5 orders of magnitude or OOMs/year)
0.5 orders of magnitude is about a factor of 3. We're growing by a factor of 3 in compute every year? I find that rather hard to believe.
Oh, echelon points out the quote:
> Over the past year, the talk of the town has shifted from $10 billion compute clusters to $100 billion clusters to trillion-dollar clusters.
That's how they're getting a factor of 3 in compute every year - by spending more money every year. That's... probably not sustainable for four more years. If it is, it's not sustainable much past that. You're probably not going to see a ten-trillion-dollar cluster; you're definitely not going to see a one-hundred-trillion-dollar cluster, ever.
> That doesn’t require believing in sci-fi; it just requires believing in straight lines on a graph.
I don't believe in straight lines on a graph. Or rather, I believe that lines are straight over a short enough time span, but may not continue to be straight for, say, twice that time span.
Or, they may stay straight for that long. But I don't rate the plausibility as high as he does. (Especially on the "spend more to increase compute" front.)
>That's... probably not sustainable for four more years. If it is, it's not sustainable much past that. You're probably not going to see a ten-trillion-dollar cluster; you're definitely not going to see a one-hundred-trillion-dollar cluster, ever.
Compute is just one part, as he claims. 3^4 is just 81X, i.e. 10B to 40B of compute, certainly within the budget of a single big tech company.
He couples compute gains with algorithmic advances, which he claims there is also a trend and with other gains(gpt to chatgpt advancements, inference time gains from searching as in alphago etc) and calls it effective compute and it might increase 5 OOM in 2027.
No, my bad. When you said "10B to 40B of compute", I read that as "growing from 10B to 40B of compute", that is, a 4X increase. I failed to understand it as "growing by 81X from current compute, which would put you at about 10B to 40B".
I don't know what current GPT-4 compute is. I don't know if your assumption is correct, but I certainly do not know that it is wrong.
$100 trillion is the entire world GDP. I don't care how much debt you're willing to take on or how much stock you sell, you're not going to take the entire world output.
What percent of world GDP do you think is realistic to capture for this? As a ballpark estimate, I would say 5% - and even that is only realistic if you're the only one trying to do it. But even at that rate, it's going to take 20 years to build it. Well, if it takes 20 years to build it and then you get a return in 7 years, will the money be there to do it? A 27-year return is a lot different from a 7-year return.
If there is ROI for $100 Trillion, we (banks) will print $100 Trillion by a click of a button
Thought Experiment. If by a magic wand the world's productivity doubles overnight. How much extra money needs to be printed to maintain price levels?
Ding, Ding, Ding.... $100 Trillion (yes, there is velocity of money and stuff).
But, if a project has positive NPV, we can print money (and it's the right thing to do).
The worst investors are people who don't get this simple concept and are scared by Debt and Inflation
> If there is ROI for $100 Trillion, we (banks) will print $100 Trillion by a click of a button
And that will create exactly zero new hardware! All it will do is create twice as much money, so everything will now cost twice as much.
As you say, money is an abstract concept. So don't think about money except as a measuring stick for stuff. Is there enough hardware for a cluster that measures at $100 trillion? No, there isn't. There won't be four years from now, or eight years from now. There's not enough hardware.
Printing an additional $100 trillion changes the length of the measuring stick, but it doesn't give us any more hardware.
> Printing an additional $100 trillion changes the length of the measuring stick, but it doesn't give us any more hardware.
It actually does. Spending freshly conjured money on hardware immediately kicks off production of new hardware and loads all the supply chain. Yes, this injection of new money into economy does eventually result in the inflation, but that will increase the costs of everything later. This lag between when you start spending and the adverse effects of inflation when the money will trickle down everywhere is the period when you actually gain from issuing new money, and the greater the economy is, the longer this lag will be.
Also, inflation is not guaranteed at all. Let's say this hardware results in robots, which can produce near infinite amount of goods and services. That immediately puts downward pressure on prices.
Not all printing money results in inflation. If you print money just about the same rate of productivity gain, you will not get inflation at all
Nope, you have a fundamental misunderstanding of economics.
Thought Experiment : Let's say the world unemployment is 25% and we can build a magical nuclear reactor that produces near infinite clean energy, but it costs $50 Trillion to build.
Are you going to wait for the world to save $50 Trillion (which may take 50 years) to build this reactor or are you going to print $50 Trillion give those 25% jobs in creating the nuclear reactor?
The only precious commodity in the universe is time. If you don't print $50 T, you are wasting the time of 25% unemployed people.
We don't have 25% unemployment among those able to build AI hardware. We don't have 25% unemployment among those able to grow pure silicon crystals, or able to create semiconductor fabs.
There are actual capacity constraints. You're not going to change them very much by printing a ton of money.
If you print money and give it to trainers / mentors so that they can scale themselves to train more people for relevant skills.
Even in US, labor participation is 62%. So we have 38% slack. Now, think about India, Africa were able bodied and smart people don't have opportunities to use their body/mind and a significant portion.
Like I said, your understanding of economics is still basic.
tl;dr -- Unless all 8 Billion people are at peak productive capacity, there is always slack and wherever there is slack printing money (with the right incentives) is an easy solution
Well, I don't think much of your understanding of economics either. 62% labor participation rate is massively different from 38% slack. Not all of those 38% are available to work. Far from it. We might have 5% slack. Pessimistically, 3%.
So back to the original point: No, we can't produce all that hardware, even if your vision of the payback is real.
Now, if it's not real, then you just upended the entire economy, caused inflation, wasted a huge amount of resources and time, all to chase an illusion.
Look, I know you believe the payoff is there. I don't. In reality, the probability has to be something less than 100% (and something more than 0%). And politically, nobody cares what either you nor I think. They only care what the population as a whole thinks, and the population as a whole does not thing that we should harness the entire global economy to chase this. So in practice, you're left with what a company can do. And, to return to my point several comments ago, a company can't do a $100 trillion cluster, and won't be able to.
Are all 8 Billion people at their peak productivity? No.
There is slack in the economy.
If people don't want to work, it's a money printing problem. At constant $$ of say $1 Million annual salary, a lot more than 3% of the 38% slack will work.
The reason we don't pay $1 Million is because we don't think they will produce $1 Million worth of output. But, if we know that we are producing $100 T output, you'll pay them $1 M Dollars
I share the authors view that if things align and compound we are in for a very wild ride.
However, I feel the bull case is over emphasized.
Not all likely "walls" are sufficiently discussed, some that I think about:
- will AI blow past human general intelligence or level off? (because of diminishing returns and "weak" human teachers)
- can AI really achieve online learning (economically)? the current architecture can't (just shoving things into large context windows doesn't seem to be enough)
- do we as a society tolerate this rapid change or will there be interventions? (protest because people don't want to lose their jobs)
The explanation without is that the LLM takes the prompt text, applies a math formula to produce the next token, iterates that function to get a set of tokens, and returns the response. Where is there reasoning?
If that is sufficient for reasoning, is my calculator reasoning when I input 2+2 and it returns 4?
If you were asked to make an HTML button that looks like a watermelon, you would start by considering what a watermelon looks like: a green shell, a red interior, and a circular cross-section. You would then take this information and apply it when writing the CSS for your button.
What word would you use to describe this process?
>The explanation without is that the LLM takes the prompt text, applies a math formula to produce the next token, iterates that function to get a set of tokens, and returns the response. Where is there reasoning?
The "formula" is dependent on prior knowledge about the world, which is used to successfully solve a problem that does not appear in the training data. Why assume that reasoning is somehow impervious to mathematical modelling?
Sure, that’s essentially what a person would do. Describing that process says nothing about what an LLM does, which we know by definition of the system is applying a set of weights to a text string input.
If you want to reduce all reasoning to just math, then LLMs reason. But then saying something like the OP of “we’ve made computers that reason” is not very novel or useful. Under that framework, computers always reasoned.
In this case, it's more like "find a bunch of images that have been associated with the word 'watermelon', then generate some pixels that kind of look like those images". That's not a very high level of reasoning.
[Edit: Though I guess it's somewhat decent to realize that, given the request for a button that "looks like X", it needs to go find images of X.]
reasoning implies some kind of logic, evaluating branching paths of abstract assumptions to draw conclusions.
sure you can argue choosing green instead of blue as the button color is a kind of reasoning, but that's too similar to memorized associations to count IMO.
It's not just him though. His views are very common in AI labs, who actually build these systems and have better visibility than the rest of us on what's going on.
Important context, this appears to be a brief intro+table of contents for a 165-page pdf/html book.[0]
It’ll probably be easy for HN to take potshots at snippets of the intro, but the writing intrigued me enough that I want to at least skim over the whole thing.
I think the website submission is the most optimal for readability. Nice and clean design too, and you can easily navigate the chapters. I don’t mind a PDF (and it is linked on the site), but in this case the web experience feels best.
Yeah, I agree! Knowing how people respond, I thought it was useful to add the full context of the breadth and depth of the author’s writing. Already there’s people pulling single line quotes to dismiss. But, that’s pretty typical of the internet.
No it's not. GPT-4 is not a "smart high-schooler". This is so hype-y, it's ridiculous.
> Over the past year, the talk of the town has shifted from $10 billion compute clusters to $100 billion clusters to trillion-dollar clusters.
Talk amongst the memesters. This is a meme.
I work in AI, and I'm building an AI animation company. But let's be realistic -- claims like these are not grounded and are going to result in distrust from stakeholders: investors, policy makers, and the general public.