LLMs have an astonishingly general set of emergent skills, which they can combine to solve novel problems (intelligence). LLMs smoke humans in many broad intellectual domains, are competitive in others, and fail abysmally in some. Folks try skirting this messy reality, squaring the circle, by conflating AGI with ASI or some techno-godhood outcome. We should evaluate AGI the same way we evaluate human general intelligence, which also manifests with dizzying variety.

This is what the "statistical parrot" (i.e: 'No True Scotsman') critics miss. Their handwavy statements about LLMs not "really" understanding the world, "only" doing simple token prediction, having no autonomy/drive, etc. miss the point: it doesn't matter. The damn thing communicating with you (on HN or elsewhere!) may display broad intelligence. I think that's why Turing's test requires a text interface -- people can't help themselves from judging agents by silly anthropomorphic heuristics (appearance, similarity to me, spacial behaviors, etc). So you must literally wall off those biases.

AGI is here. It just looks inhuman and our GI is "better", for now.

This is key. All humans don’t understand everything—and the way we assess human conceptual understanding is through tests. This implies that we need a lot more tests for AI.

I’d like to see a big investment in “Machine Psychology.”