I don't know. People in the 90s were initially fooled by Eliza, but soon understood that Eliza was a trick. LLMs are a more complex and expensive trick. Maybe it's time to overthrow the Turing Test. Fooling humans isn't necessarily an indicator of intelligence, and it leads down a blind alley: Language is a false proxy for thought.
Consider this. I could walk into a club in Vegas, throw down $10,000 cash for a VIP table, and start throwing around $100 bills. Would that make most people think I'm wealthy? Yes. Am I actually wealthy? No. But clearly the test is the wrong test. All show and no go.
The more I think about this, the more I think the same is true for our own intelligence. Consciousness is a trick and AI development is lifting the veil of our vanity. I'm not claiming that LLMs are conscious or intelligent or whatever. I'm suggesting that next token prediction has scaled so well and cover so many use cases that the next couple breakthroughs will show us how simple intelligence is once you remove the complexity of biological systems from the equation.
All we know about animal consciousness is limited to behaviour, e.g. the subset of the 40 or so "consciousness" definitions which are things like "not asleep" or "responds to environment".
We don't know that there's anything like our rich inner world in the mind of a chimpanzee, let alone a dog, let alone a lobster.
We don't know what test to make in order to determine if any other intelligence, including humans and AI, actually has an inner experience — including by asking, because we can neither be sure if the failure to report one indicates the absence, nor if the ability to report one is more than just mimicking the voices around them.
For the latter, note that many humans with aphantasia only find out that "visualisation" isn't just a metaphor at some point in adulthood, and both before and after this realisation they can still use it as a metaphor without having a mind's eye.
> Language is the baseline to collaboration - not intelligence
Would you describe intercellular chemical signals in multicellular organisms to be "language"?
> We don't know that there's anything like our rich inner world in the mind of a chimpanzee, let alone a dog, let alone a lobster.
If be "we don't know" you mean we cannot prove, then, sure, but then we don't know anything aside from maybe mathematics. We have a lot of evidence that animals similar consciousness as we do. Dolphins (or whales?) have been known to push drowning people to the surface like they do for a calf. Killer whales coordinate in hunting, and have taken an animus to small boats, intentionally trying to capsize it. I've seen squirrels in the back yard fake burying a nut, and moving fallen leaves to hide a burial spot. Any one who has had a dog or a cat knows they get lonely and angry and guilty. A friend of mine had personal troubles and abandoned his house for a while; I went over to take pictures so he could AirBnB it, and their cat saw me in the house and was crying really piteously, because it had just grown out of being a kitten with a bunch of kids around and getting lots of attention, and suddenly its whole world was vanished. A speech pathologist made buttons for her dog that said words when pressed, and the dog put sentences together and even had emotional meltdowns on the level of a young child. Parrots seem to be intelligent, and I've read several reports where they give intelligent responses (such as "I'm afraid" when the owner asked if it wanted to be put in the same room as the cat for company while the owner was away [in this case, the owner seems to be lacking in intelligence for thinking that was a good idea]). There was a story linked her some years back about a zoo-keeper who had her baby die, and signed it to the chimpanzee (or gorilla or some-such) females when it wanted to know why she had been gone, and in response the chimpanzee motioned to with its eye suggesting crying, as if asking if she were grieving.
I probably have some of those details wrong, but I think there definitely is something there that is qualitatively similar to humans, although not on the same level.
> If be "we don't know" you mean we cannot prove, then, sure, but then we don't know anything aside from maybe mathematics.
More than just that: we don't know what the question is that we're trying to ask. We're pre-paradigmatic.
All of the behaviour you list, those can be emulated by an artificial neural network, the first half even by a small ANN that's mis-classifying various things in its environment — should we call such an artificial neural network "conscious"? I don't ask this as a rhetorical device to cast doubt on the conclusion, I genuinely don't know, and my point is that nobody else seems to either.
> We don't know that there's anything like our rich inner world in the mind of a ...
I posit that we should start with a default "this animal experiences the world the same as I do" until proven differently. Doctors used to think human babies could not feel pain. The assumption has always been "this animal is a rock and doesn't experience anything like me, God's divine creation." It was stupid when applied to babies. It is stupid when applied to animals.
Did you know that jumping spiders can spot prey, move out of line of sight, approach said pray outside that specific prey's ability to detect, and then attack? How could anything do that without a model of the world? MRIs on mice have shown that they plan and experience actions ahead of doing them. Just like when you plan to throw a ball or lift something heavy where you think through it first. Polar bears will spot walrus, go for a long ass swim (again, out of sight) and approach from behind the colony to attack. A spider and the apex bear have models of the world and their prey.
Show that the animal doesn't have a rich inner world before defaulting to "it doesn't."
> I posit that we should start with a default "this animal experiences the world the same as I do" until proven differently.
As I don't know, I take the defensive position both ways for different questions.*
Just in case they have an inner world: We should be kind to animals, not eat them, not castrate them (unless their reproductive method appears to be non-consensual), not allow them to be selectively bred for human interest without regard to their own, etc.
I'd say ditto for AI, but in their case, even under the assumption that they have an inner world (which isn't at all certain!), it's not clear what "be kind" even looks like: are LLMs complex enough to have created an inner model of emotion where getting the tokens for "thanks!" has a feeling that is good? Or are all tokens equal, and the only pleasure-analog or pain-analog they ever experienced were training experiences to shift the model weights?
(I'm still going to say "please" to the LLMs even if it has no emotion: they're trained on human responses, and humans give better responses when the counterparty is polite).
> How could anything do that without a model of the world?
Is "a model of the world" (external) necessarily "a rich inner world" (internal, qualia)? If it can be proven so, then AI must be likewise.
* The case where I say that the defensive position is to say "no" is currently still hypothetical: if someone is dying and wishes to preserve their continuity of consciousness, is it sufficient to scan their brain** and simulate it?
There are some clever tests described in The Language Puzzle on primates that (paraphrasing 14 hour long audiobook so forgive any mistakes.) indicate no primate other than humans and a couple of immediate predecessors (based on archaeological evidence) have much in the realm of abstract thinking abilities using their own communications, a few primates raised and taught forms of human language cannot progress very far without any of the facilities of language present in normal two-three year old development. The book is focused on how humans evolved language so other species are not covered, there is obvious verbal and gesture based communication in primates but it concludes not enough of the components of physiology that enable human language are present(both brain and vocal anatomy).
How do you define verbal language? Many animals emit different sounds that others in their community know how to react to. Some even get quite complex in structure (eg dolphins and whales) but I wouldn’t also rule out some species of birds, and some primates to start with. And they can collaborate; elephants, dolphins, and wolves for example collaborate and would die without it.
Also it’s completely myopic in terms of ignoring humans who have non verbal language (eg sign language) perfectly capable of cooperation.
TLDR: just because you can’t understand an animal doesn’t mean it lacks the capability you failed to actually define properly.
MW defines verbal as "of, relating to, or consisting of words".
I don't think anyone would argue that animals don't communicate with each other. Some may even have language we can't interpret, which may consist of something like words.
The question is why we would model an AGI after verbal language as opposed to modeling it after the native intelligence of all life which eventually leads to communication as a result. Language and communication is a side-effect of intelligence, it's a compounding interest on intelligence, but it is not intelligence itself, any more than a map is the terrain.
> The question is why we would model an AGI after verbal language as opposed to modeling it after the native intelligence of all life which eventually leads to communication as a result.
Because verbal/written language is an abstracted/compressed representation of reality, so it's relatively cheap to process (a high-level natural-language description of an apple takes far fewer bytes to represent than a photo or 3D model of the same apple). Also because there are massive digitized publicly-available collections of language that are easy to train on (the web, libraries of digitized books, etc).
I'm just answering your question here, not implying that language processing is the path towards AGI (I personally think it could play a part, but can't be anything close to the whole picture).
This is one of the last bastions of anthropocentric thinking. I hope this will change in this century. I believe even plants are capable of communication. Everything that changes over time or space can be a signal. And most organisms can generate or detect signals. Which means they do communicate. The term “language” has traditionally been defined from an anthropocentric perspective. Like many other definitions about the intellect (consciousness, reasoning etc.).
That’s like a bird saying planes can’t fly because they don’t flap their wings.
LLMs use human language mainly because they need to communicate with humans. Their inputs and outputs are human language. But in between, they don’t think in human language.
> LLMs use human language mainly because they need to communicate with humans. Their inputs and outputs are human language. But in between, they don’t think in human language.
You seem to fundamentally misunderstand what llms are and how they work, honestly.
Remove the human language from the model and you end up with nothing. That's the whole issue.
Your comment would only make sense if we had real artificial intelligence, but LLMs are quite literally working by predicting the next token - which works incredibly well for a fascimlie of intelligence because there is an incredible amount of written content on the Internet which was written by intelligent people
A human child not taught literally anything can see some interesting item extend a hand to it, touch it, interact with it. All decided by the child. Heck, even my cat can see a new toy, go to it and play with it, without any teaching.
LLMs can't initiate any task on their own, because they lack thinking/intelligence part.
This to me overstretches the definition of teaching. No, a human baby is not "taught" language, it learns it independently by taking cues from its environment. A child absolutely comes with an innate ability to recognize human sound and the capability to reproduce it.
By the time you get to active "teaching", the child has already learned language -- otherwise we'd have a chicken-and-egg problem, since we use language to teach language.
Transformers are very powerful also for non-language data. For example time series, sequences like DNA or audio (also outside of speech and music).
Of course the vast amount of human text is key to training a typical LLM, but it is not the only use.
>but LLMs are quite literally working by predicting the next token - which works incredibly well for a fascimlie of intelligence because there is an incredible amount of written content on the Internet which was written by intelligent people
An additional facet nobody ever seems to mention:
Human language is structured, and seems to follow similar base rules everywhere.
That is a huge boon to any statistical model trying to approximate it. That's why simpler forms of language generation are even possible. It's also a large part of why LLMs are able to do some code, but regularly fuck up the meaning when you aren't paying attention. The "shape" of code and language is really simple.
How do we know animal language isn’t structured, in similar ways? For example we now know that “dark” birds are often colorful, just in the UV spectrum they can see and we can’t. Similarly there’s evidence dolphin and whale speech may be structured, we just don’t know the base rules; their speech is modulated at such rapid frequency our computers until maybe recently would struggle to even record and process that data realtime (probably still do).
Just because we don’t understand something doesn’t mean there’s nothing there.
Also, I’m not so sure human language is structured the same way globally. There’s languages quite far from each other and the similarities tend to be grouped by where the languages originated. Eg Spanish and French might share similarities of rules, but those similarities are not shared with Hungary or Chinese. There’s cross pollination of course but language is old and humans all come from a single location so it’s not surprising for there to be some kinds of links but even a few hundred thousand years of evolution have diverged the rules significantly.
Well, you can explain to a plant in your room that E=mc2 in a couple of sentences, a plant can't explain to you how it feels the world.
If cows were eating grass and conceptualising what is infinity, and what is her role in the universe, and how she was born, and what would happen after she is dead... we would see a lot of jumpy cows out there.
This is exactly what I mean by anthropocentric thinking. Plants talk plant things and cows talk about cow issues. Maybe there are alien cows in some planet with larger brains and can do advanced physics in their moo language. Or some giant network of alien fungi discussing about their existential crisis. Maybe ants talk about ant politics by moving their antennae. Maybe they vote and make decisions. Or bees talk about elaborate honey economics by modulating their buzz. Or maybe plants tell bees the best time for picking pollens by changing their colors and smell.
Words, after all are just arbitrary ink shapes on paper. Or vibrations in air. Not fundamentally different than any other signal. Meaning is added only by the human brain.
I'm also attracted to the idea of reducing rule sets to simple algorithms and axioms, in every case you can. But I'm skeptical that consciousness can be reduced that way. I think if it can be, we'll see it in the distillation and quantizing of smaller and smaller scale models converging on similar adaptations, as opposed to the need for greater scale (at least in inference). I still believe language processing is the wrong task to train to that point. I'd like to see AIs that model thought process, logic, tool construction, real-world tasks without language. Maybe even those that model vocal chords and neurological processes instead of phonemes. Most animals don't use language, and as a result we can't ask if they're conscious, but they probably are. Navigating and manipulating the physical world from the cellular level up to swinging from trees is far more complex - language is a very late invention, and is not in and of itself intelligence - it may just be a lagging indicator.
To the extent that we vainly consider ourselves intelligent for our linguistic abilities, sure. But this underrates the other types of spatial and procedural reasoning that humans possess, or even the type that spiders possess.
That's not how I view it. Consciousness is the result of various feedback structures in the brain, similar to how self-awareness stems from the actuator-sensor feedback loop of the interaction between the nervous system and the skeletomuscular system. Neither of those two definitions have anything to do with language ability -- and it bothers me that many people are so eager to reduce consciousness to programmed language responses only.
The validity of the Turing test doesn’t change the fact that the bots are better than humans at many tasks that we would consider intellectual challenges
I am not a good writer or artist, yet I can tell that AI generated pictures or prose feel 'off' compared to stuff that humans make. People who are professional writers and artists can point out in a lot of cases the issues with structure, execution and composition that these images have, or maybe if sometimes they can't they still have a nose for subtle issues, and can improve on the result.
>I could walk into a club in Vegas, throw down $10,000 cash for a VIP table, and start throwing around $100 bills.
If you can withdraw $10,000 cash at all to dispose as you please (including for this 'trick' game) then my friend you are wealthy from the perspective of the vast majority of humans living on the planet.
And if you balk at doing this, maybe because you cannot actually withdraw that much, or maybe because it is badly needed for something else, then you are not actually capable of performing the test now, are you ?
That's really not true. Lots of people in America can have $0 in net worth and get a credit card, use that to buy some jewelry and then sell it, and have $10k in cash. The fact that the trick only works once proves that it's a trick.
You're not making much sense. Like the other user, you are hinging on non-transferrable details of your analogy, which is not the actual reality of the situation.
You've invented a story where the user can pass the test by only doing this once and hinged your point on that, but that's just that - a story.
All of our tests and benchmarks account for repeatability. The machine in question has no problem replicating its results on whatever test, so it's a moot point.
The LLM can replicate the trick of fooling users into thinking it's conscious as long as there is a sufficient supply of money to keep the LLM running and a sufficient number of new users who don't know the trick. If you don't account for either of those resources running out, you're not testing whether its feats are truly repeatable.
>The LLM can replicate the trick of fooling users into thinking it's conscious as long as there is a sufficient supply of money to keep the LLM running and a sufficient number of new users who don't know the trick.
Okay ? and you, presumably a human can replicate the trick of fooling me into thinking you're conscious as long as there is a sufficient supply of food to keep you running. So what's your point ? With each comment, you make less sense. Sorry to tell you, but there is no trick.
The difference is that the human can and did find its own food for literally ages. That's already a very, very important difference. And while we cannot really define what's conscious, it's a bit easier (still with some edge cases) to define what is alive. And probably what is alive has some degree of consciousness.
An LLM definitely does not.
One of the "barriers" to me is that (AFAIK) an LLM/agent/whatever doesn't operate without you hitting the equivalent of an on switch.
It does not think idle thoughts while it's not being asked questions. It's not ruminating over its past responses after having replied. It's just off until the next prompt.
Side note: whatever future we get where LLMs get their own food is probably not one I want a part of. I've seen the movies.
You only exist because you were forced to be birthed externally? Everything has a beginning.
In fact, what is artificial is stopping the generation of an LLM when it reaches a 'stop token'.
A more natural barrier is the attention size, but with 2 million tokens, LLMs can think for a long time without losing any context. And you can take over with memory tools for longer horizon tasks.
>All of our tests and benchmarks account for repeatability.
What does repeatability have to do with intelligence? If I ask a 6 year old "Is 1+1=2" I don't change my estimation of their intelligence the 400th time they answer correctly.
>The machine in question has no problem replicating its results on whatever test
What machine is that? All the LLMs I have tried produce neat results on very narrow topics but fail on consistency and generality. Which seems like something you would want in a general intelligence.
>What does repeatability have to do with intelligence? If I ask a 6 year old "Is 1+1=2" I don't change my estimation of their intelligence the 400th time they answer correctly.
If your 6 year old can only answer correctly a few times out of that 400 and you don't change your estimation of their understanding of arithmetic then, I sure hope you are not a teacher.
>What machine is that? All the LLMs I have tried produce neat results on very narrow topics but fail on consistency and generality. Which seems like something you would want in a general intelligence.
No LLM will score 80% on benchmark x today then 50% on the same 2 days later. That doesn't happen, so the convoluted setup OP had is meaningless. LLMs do not 'fail' on consistency or generality.
>Couldn’t someone else just give him a bunch of cash to blow on the test, to spoil the result?
If you still need a rich person to pass the test, then the test is working as intended. Person A is rich or person A is backed by a rich sponsor is not a material difference for the test. You are hinging too much on minute details of the analogy.
In the real word, your riches can be sponsored by someone else, but for whatever intelligence task we envision, if the machine is taking it then the machine is taking it.
>Couldn’t he give away his last dollar but pretend he’s just going to another casino?
Again, if you have $10,000 you can just withdraw today and give away, last dollar or not, the vast majority of people on this planet would call you wealthy. You have to understand that this is just not something most humans can actually do, even on their deathbed.
>> Again, if you have $10,000 you can just withdraw today and give away, last dollar or not, the vast majority of people on this planet would call you wealthy. You have to understand that this is just not something most humans can actually do, even on their deathbed.
So, most people can't get $1 Trillion to build a machine that fools people into thinking it's intelligent. That's probably also not a trick that will ever be repeated.
>> I should be able to get one up and running for you by the middle of next year
Funny. I agree with your plainspoken analysis of why these things are nowhere near AGI, and of what AGI would be. I even had a long conversation with Claude last week where it told me that no LLM would ever approach AGI (but then it wrote a 4-paragraph-long diatribe entitled "Why I Declare Myself Conscious" in the same conversation). These neural networks are closer to the speechwriting machine in The Penultimate Truth, or the songwriting machine in 1984. As for that latter one, I believe Orwell remarks on how it just recycles the same sentimental tunes and words in different order so that there's always a "new" song all the proles are humming.
Consider this. I could walk into a club in Vegas, throw down $10,000 cash for a VIP table, and start throwing around $100 bills. Would that make most people think I'm wealthy? Yes. Am I actually wealthy? No. But clearly the test is the wrong test. All show and no go.