I was thinking that too. I wouldn’t even trust that the “offline” tests didn’t have the questions and answers posted online somewhere. This might really be an analysis of how extensive the dataset is for each LLM, not how much smarter one LLM is from another.
A compromise would be to double the undergrad class size while limiting legacy to something less than or equal to what it is today in absolute terms. Many more deserving students would get and Stanford would get to keep its cash cow. But of course that would entail Palo Alto to let it expand, which it very much wants to do. And good luck with that.
I’ve known some people who didn’t want to learn the syntax of numpy and did it all in loops, and the code was not easy to read. It was harder to read. The fundamental issue is that operations on high dimensional arrays are very difficult to reason about. Numpy can probably be improved, but I don’t think loops are the answer.
The point here is not that it’s loops per se, the point is that the indexing is explicit. It seems like a big win to me. The article’s ~10 non-trivial examples all make the code easier to read, and more importantly, to understand exactly what the code is doing. It is true that some operations are difficult to reason about, that’s where explicit indexing really helps. The article resonates with me because I do want to learn numpy syntax, I’ve written hundreds of programs with nympy, spent countless hours doing battle with it, and I feel like I’m no better off now than someone who’s brand new to it. The indexing is constantly confounding, nothing ever just works. Anytime you see “None” and “axis=“ inside an operation, it’s a tell: bound to be difficult to comprehend. I’m always having to guess how to use some combination of reshape, dstack, hstack, transpose, and five other shape changers I’m forgetting, just to get something to work and it’s difficult to read and understand later. It feels like there is no debugging, only rewriting. I keep reading the manual for einsum over again and I’ve used it, but I can’t explain how, why, or when to use it, it seems like this thing you have to resort to because no other indexing seems to work. The ability to do straightforward explicit non-clever indexing as if you were writing loops seems like a pretty big step forward.
I involuntarily whispered "reshape" to myself near the top of your comment. Numpy is a very different way for me to think and I have similar feelings to what you're describing.
I could never understand why people use dstack, hstack and the like. I think plain np.stack and specifying the axis explicitely is easier to write and to read.
For transposes, np.einsum can be easier to read as it let's you use (single character, admittedly) axis specifiers to "name" them.
The real question—to which I have absolutely no answer—is not about syntax, it's about concepts: what is a better way to think about higher-dimensional arrays rather than loops and indices? I'm convinced that something better exists and, if it existed, encoding it in a sufficiently expressive (ie probably not-Python) language would give us the corresponding syntax, but trying to come up with a better syntax without a better conceptual model won't get us very far.
Then again, maybe even that is wrong! "Notation as a tool for thought" and all that. Maybe "dimension-munging" in APL really is the best way to do these things, once you really understand it.
Numpy seems somewhat constrained here… it grew out of the matrix ecosystem, and matrices map naturally to two-dimensional arrays (sidenote: it’s super annoying that we have n-dimensional matrices and n-dimensional arrays, but the matrix dimension maps to the width of the array).
Anyway, the general problem of having an n-dimensional array and wanting to dynamically… I dunno, it is a little tricky. But, sometimes when I see the examples people pop up with, I wonder how much pressure could be relieved if we just had a nice way of expressing operations on block or partitioned matrices. Like the canonical annoying example of wanting to apply solve using a series of small-ish matrices on a series of vectors, that’s just a block diagonal matrix…
English. "Write me a Python function or program that does X, Y, and Z on U and V using W." That will be the inevitable outcome of current trends, where relatively-primitive AI tools are used to write slightly more sophisticated code than would otherwise be written, which in turn is used to create slightly less-primitive AI tools.
For example, I just cut-and-pasted the author's own cri de coeur into Claude: https://claude.ai/share/1d750315-bffa-434b-a7e8-fb4d739ac89a Presumably at least one of the vectorized versions it replied with will work, although none is identical to the author's version.
When this cycle ends, high-level programs and functions will be as incomprehensible to most mainstream developers as assembly is today. Today's specs are tomorrow's programs.
Not a bad thing, really. And overdue, as the article makes all too clear. But the transition will be a dizzying one, with plenty of collateral disruption along the way.
I’m curious if any Trump voters are starting to feel any buyer’s remorse. The pain from the tariffs is only going to get worse. Soon we will have Trumpflation and a Trumpcession. Meanwhile, auto manufacturers are laying off employees. We are seeing pain with no gain. There is no reindustrialization nor will there ever be through this policy. For every economist except for Navarro, this is quite the own goal. I’m guessing you all were expecting Trump 1. Any doubts creeping in yet?
Never underestimate the power of shame on the human psyche. Many would rather double down on the "reality distortion field" than to admit wrongdoing or poor judgement.
Somebody has to manage the business relationships and go out and sell. Somebody has to be in charge of marketing, payroll, and PR. Somebody has to be the public face of the company, out networking and attending trade shows. If you’re a technical founder and all this sounds like fun to you, then yeah, a non-technical cofounder would not offer much. But there’s something to be said for having more time to put your head down and build.
Well said. If the company is built around a certain specialism, the specialist will want to spend most of your time there, because that is where most value is created per hour. But many of the other things are critical, but happen to be very time-consuming. Such as traveling around. So in some teams you end up having one person do this specialist work while the other can focus on the more time-consuming things. I think you worded it much more elegantly, I'm just repeating what you said.
Given that o3 is trained on the contents of the Internet, and the answers to all these chess problems are almost certainly on the Internet in multiple places, in a sense it has been weakly trained on this content. The question for me becomes: is the LLM doing better on these problems because it’s improving in reasoning, or is it simply improving in information retrieval.
And then there's the further question of where we draw the line in ourselves. One of my teachers -- a philosopher -- once said that real, actual thought is incredibly rare. He's a world-renowned expert but says he can count on one hand the number of times in his life that he felt he was thinking rather than remembering and reorganizing what he already knew.
That's not to say "are you remembering or reasoning" means the same thing when applied to humans vs when it's applied to LLMs.
I had a similar experience where math went from being easy and fun to an intimidating and painful slog. My problem was just how focused most courses are on learning techniques for solving problems. I found all those endless substitutions that you learn in calculus to be infinitely dull and so it was difficult to do a good job. Ditto for the solution techniques for differential equations. Don’t get me started on matrix inverting. I think I had to do a 5x5 matrix once for a homework assignment. What a colossal waste of time.
Proof-based math classes came like a revelation to me. When I took Real Analysis, for the first time in over a decade, math was fun. You weren’t just memorizing and reapplying recipes. You were seriously thinking about unique problems and devising solutions. And all the while, you were learning where all these techniques actually came from and how everything connected together.
I don’t understand why we can’t have more proof heavy math in high school. Who cares whether you remember the arctan substitution or whatever in an integral; I’d always just use a solver for that anyway. I’d rather be learning about what an integral is in the first place.
I'm a private tutor who works with adults on proof-based math. I've often had a similar thought to the one you're expressing here --- I also found proofs pretty revelatory when I first exposed to them and wondered where this magical tool had been all my life --- but I wonder how well this experience would scale to the mass of students in high school math classes.
After teaching proof-writing to my students for several years now, I've seen a lot of variation in how quickly students take to the skill. Some of them have the same experience that it sounds like you and I had, where it "clicks" right away, some of them struggle for a while to figure out what the whole enterprise is even about, and everything in between. Basically everyone gets better at it over time, but for some that can mean spending a decent amount of time feeling kind of lost and frustrated.
And this is a very self-selected group of students: they're all grown-ups who decided to spend their money and spare time learning this stuff in addition to their jobs! For the kind of high school student who just doesn't really think of themselves as a "math person", who isn't already intrinsically motivated by the joy of discovering what makes integrals tick, I think it would be an even harder sell. High school math teachers have a hard job: they have to try to reach students at a pretty wide range of interest and ability levels, and sadly that often leads to a sort of lowest-common-denominator curriculum that doesn't involve a lot of risk-taking.
A more rigourous approach was tried after WW2, when Americans feared the Soviets were edging ahead mathematically/scientifically. It was called "New Math" [0]. For an example of the type of textbook high school students were taught from, check out Dolciani's Modern Introductory Analysis (the 1960s and 1970s editions only; the later editions were dumbed down, especially when Dolciani died) [1], which starts with set theory, logic, field axioms, and proof writing techniques.
> I don’t understand why we can’t have more proof heavy math in high school.
proof based math requires critical thinking and its a lot harder to scale the teaching of critical thinking. We dont' pay enough for teachers of quality to be able to do this at the public school level. Its also much harder to test for in standardized tests.
> Its also much harder to test for in standardized tests.
You could test it using interactive proof verifiers. This would also make it a lot easier to teach, since proof verifiers can handle even very complex mathematical proof via the repeated application of a mere handful of rules. (The rules are also surprisingly similar to the familiar "plug and chug" workflow of school-level math, only with different underlying objects - lemmas and theorems as opposed to variables and expressions.)
I disagree. Before 538 people were still offering lots of election predictions and it was much much worse, because it was based entirely on hunches and vibes. Silvers rates the pollsters and provides confidence intervals far better than a simple average of polls does. I’d much rather read his forecasts than any number of bloviating opeds.
I’m sad to say that you are the one who does not understand urban development. There’s actually quite a lot of research on induced demand (which is the name for what you’re talking about) and there isn’t any evidence of it in the housing market. Rather, new construction lowers rents across the board. Which makes sense. If a bunch of new people want to live in a place and no new housing is made for them, what else can happen but a rise in prices?
Here’s just a taste. Do please read and if you have a problem with the identification strategies, I would love to hear it.
Seriously, all of the cities are massively overbuilt. More so the smaller ones, but even Shanghai, Beijing, Shenzhen, Chongqing. AirBnB is not allowed. Huge numbers of units sit never used and off market. BTW that's with a SHRINKING population. Overbuild estimate is 50M units (150M people) with some 20M units unfinished. Prices are insane and comparable to NYC/London @ $10k/m2, when incomes are only 30% as high.
Why? Financialization. Yes, property values ONLY started falling WHEN builders stopped completing buildings. They were not places to live, they were investment vehicles. This is just as true in London, Vancouver, NYC, SF, Miami, etc.
A 1 bedroom in the inner ring might go for around $400/mo, and in the outer ring around $200-300/mo. Double those numbers for 2bdrm, and 3-4x those numbers for the inner ring of a T1 like Beijing, Shanghai, Shenzhen, etc.
By American standards, that might be affordable, but median household income in China is around $300-400/mo, and the urban median household income is around $500/mo.
As such, it can be fairly unaffordable, but buyers who were lucky can continue to demand high(ish) rents while coasting on asset value depending on when they acquired property - especially inner ring property.
> A property bubble generally involves asset values rising past what rents support.
Rent is often used as a proxy for spending, but that doesn't necessarily work in China.
In China, because incomes outside of a couple top tier white collar employers remain low, private rents can be fairly inaccessible for the bottom half, so employers such as large factories often provide subsidized dorms/apartments, but these factory cities tend to be isolated and self contained from the larger community.
Furthermore, despite significant reforms, China still has a corruption problem, and the easiest way to convert black money into white money is real estate, so in that situation, asset depreciation is acceptable because it can make previously non fungible black yuan into fungible yuan or USD sellable or mortgagable assets.
This problem has constantly manifested all over Asia, and has tripped various former high flying economies like Malaysia (1990s), Thailand (1990s), India (2010s), Vietnam (present), etc.
I think it depends. In the US, denser cities are more expensive than lower density ones. If a city doubles in population, it will probably generate more demand due to the extra jobs opportunities/amenities/culture/etc, so people from other parts of the country will want to move there.
The exception is if everywhere in the country increases in size at the same time. For example, Japan has cheap rent at the most populous city at Earth (Tokyo). But one quarter of the whole country lives there.
If the US had something equivalent, with 80 million people in NYC, and plenty of high-rises and missing middle in every city that is currently mostly single-family-homes, then we could have cheap rents everywhere. But if just one city increases density alone, it seems it will just attract people from elsewhere in the country
You might be simply struggling with some cognitive hurdle here, I don't know what it is, and it's hard for me to help you.
Maybe try to research the city of Barcelona, its topology, the available space for construction, the process of developing high-rise buildings, and then try to ask yourself what steps need to be taken to build 100k new flats in the city?
Consider the infrastructure, the cost, the timeframe, steps needed to ensure quality, and make sure that it doesn't become unliveable like american downtowns, compensation to the current residents whose flats would have to be demolished, damage to the city's character...
Then you could maybe think about what happens when rent prices drop, and whether a decrease in apartment prices that follow would have some effect on the number of mortgage defaults.
Maybe consider the political support you'd need for such a development and how you'd get it from the existing residents?
Finally try to do some napkin math and figure out how quickly these new units would be bought up as vacation houses by the rest of Europe and the rest of Spain moving from smaller towns to the newly available stock in the city. For bonus points you could try to estimate how much new housing was built in Barcelona in the last 20 years.
Maybe that'll help. I'm really not sure what you mean by "identification strategies", but thanks for all your research about just letting developers build denser housing. I'm sure if they just listened to your simple advice, the problem would have been solved long ago. NYC and SF are known for their affordable and high quality rentals.
NYC and SF under build severely. Have a look at the chart of their building versus Tokyo, the only place that actually has kept a lid on rental inflation.
> You might be simply struggling with some cognitive hurdle here
Arrogance is thinking your myopic macro model has the power to fix every housing price crisis in the world. While ignoring every important factor but one.
Is a glut of English and Comparative Literature PhDs really that big of a benefit? Those skills are not transferable to anything. I think it’s a crime the way liberal arts departments admit way more students than could ever hope to find a job in Academia. I say this as someone who loves literature and is sad to see these departments shrink. But it isn’t fair to the students to put them through so much pain when you know there is nothing for them at the end of the tunnel.
In the early 2000s those liberal arts departments went as far as Southeast Asia to recruit international students who paid a lot more than domestic students, especially at the time. One of their outreach programs in Myanmar is called the Pre-Collegiate Program, whose website claims to promote critical reasoning among young people.
Except I actually spoke to several of them who said that they were heavily groomed into joining the liberal arts departments. Not one of them went into engineering or the sciences. One student said during the program she was told she "must" choose the liberal arts. Another described how he was sweet talked by a philosophy professor into becoming a philosophy major, despite having followed a science-based curriculum in high school and little-to-no education in the arts (back then they had to specialize in either but not both in high school).
So when you said "crime" I thought "funny you should say that". It might not be criminal but there was definitely some creepy stuff going on.
Not quite sure what you're talking about. The majority of PhDs awarded in the US were science and engineering (S&E) degrees. The number of non S&E PhDs has held steady since about 1973 [1].
It's also never been a 1:1 ratio of PhD recipients ending up in academia. I will agree that many universities overinflate job prospects post-graduation, but students should also be doing their own market research before entering into such a long process.
>According to data from the Survey of Earned Doctorates (SED), around 1,600 English and Literature PhDs are awarded annually in the United States. Total PhDs awarded annually in all categories : 57,862
It is a very small field that is being used as a straw-man for all PhDs. I don't know what benefit those 1,600 may produce, but I'd rather them have that expertise and use it for our country than have them leave the US for better opportunities elsewhere. Because they will leave.
The benefit is intangible and honestly if they didn't have the opportunity to pursue a PhD they wouldn't leave the country. They would do what the 99% of people like them who are unable to pursue a PhD in their chosen topic do, work another job and publish a smaller body of work in a less prestigious setting. The fact of the matter is that those are not the 1,600 people who have the ability to earn a PhD in English or Literature, it's the ones who's interests and personal profile afforded them the opportunity.
Now to be clear, I'm not saying that this work is unimportant. Intangible benefits are (despite the name) very real and do benefit the nation. It's just a much more complicated than engineering PhDs making stronger forms of concrete or whatever.
The glut is primarily a function of the fact that universities have decided that it's fine to have most of their courses taught by poorly-paid adjuncts. That is, actually, a bad thing. If we returned to having tenure-track faculty do a substantial majority of teaching, most people who get humanities degrees would get jobs in the end.
reply