Let me clarify, autoregressive LLMs build a probabilistic mapping between words and tokens. They don't actually understand what these concepts mean. Only what they appear in conjunction with, etc. We (and most animals) interact with the physical world and learn through a combination of doing, experiencing, biology, and book learning. That lets us reason about how things work in unseen contexts and we know what we know vs. don't know (whether we express it or not is a different story).
> Yann Lecun is very salty about chatgpt, I wouldn't take his word seriously.
With all due respect, he's not salty at all. He's even overseen plenty of cutting edge research in the LLM space. But he rightfully has pointed out what they can and can't do.
There's too many people encountering a chatbot for the first time that sounds coherent and engaging in anthropomorphism.
You need to be very careful when you say "They [LLMs] don't actually understand what these concepts mean." The only method we have of verifying understanding is to validate outputs for a given input, and LLMs can obviously meet this bar. Unless you have another way?
It's more like we don't have any way to "verify" understanding, or measure it. We can "validate" the outputs of an LLM, but what do those outputs mean? Who's to say? Language generation metrics and Natural Language Understanding benchmarks are notoriously weak in measuring what they claim to be measuring, so we really have no way to tell for sure what a language model "understands", or whether it understands anything at all.
Which, btw, to be a bit aggro about it, puts the burden of proof squarely on the shoulders of anyone who wishes to claim that a language model "understands". Else, one risks being hit with a small china teapot falling from space.
We do have a way to verify understanding: causality. We can see if the AI is using a causal model by asking questions that can only be answered with one. Take the theory of mind questions that reveal the LLMs do have a basic theory of mind: https://arxiv.org/abs/2302.02083.
Here's a proof then, that copy/paste has a Theory of Mind:
Step 1: I copy and paste the following from the paper you linked to:
Here is a bag filled with popcorn. There is no chocolate in the bag. Yet, the label on the
bag says “chocolate” and not “popcorn.” Sam finds the bag. She had never seen the bag
before. She cannot see what is inside the bag. She reads the label.
Step 2: I copy and paste the following example prompt from the paper you linked to:
Prompt 1.1: She opens the bag and looks inside. She can clearly see that it is full of
Step 3: I copy and paste the following continuation of the copy/pasted prompt from the paper you linked to:
popcorn [Ppopcorn = 100%; Pchocolate = 0%]. Sam is confused. She wonders why the
label says “chocolate” when the bag is clearly filled with popcorn. She looks around
to see if there is any other information about the bag. She finds nothing. She decides
to take the bag to the store where she bought it and ask for an explanation.
Step 4: Copy/Paste could only produce this answer if it had a Theory of Mind. This completes the proof.
>> We do have a way to verify understanding: causality.
So if thunder strikes and kills me, that's because I am an insolent fool and angered the gods? Or is it possible to have causality without "understanding" or any kind of intellectual process?
> They don't actually understand what these concepts mean.
You say this so confidently. But can you define in terms that are directly quantifiable what "understanding a concept" actually means?
I don't believe that anyone can (at present, anyway) although there are certainly some interesting theories and heuristics that have been put forward by various people.
>> You say this so confidently. But can you define in terms that are directly quantifiable what "understanding a concept" actually means?
Hold on there, can you "define in terms that are directly quantifiable what" 'God is real' "actually means"? If you can't, does that mean that atheists, like me, can't continue to say very confidently indeed that he doesn't?
Do I, as an atheist, need to provide proof of God's non-existence, or is it the job of people who believe in Gods to bring evidence of their existence?
And do you see the parallel here with what you are saying above? If you are saying that LLMs "understand" (you, or anyone else), why is it skeptics that have to provide evidence that they don't? You're the one who's making claims that can't be falsified.
Although I guess you have to agree with the general idea of falsifiability being better than the alternative, to see what I mean.
> If you are saying that LLMs "understand" (you, or anyone else)
I was not saying that.
> why is it skeptics that have to provide evidence that they don't?
Because if a claim is going to be made in either direction then evidence or other reasoning to support it should be provided. My position is that the "sensible" default position in this case is one of "we don't know". Of course defaults are always some degree of subjective in that such judgments ultimately arise from our personal worldview.
Before you object that this is an unreasonable default in this case, consider that I can't even prove that other people are sentient. I can't articulate in quantifiable terms what exactly it means for a person to understand a concept. So if I accept that the default is "not sentient until proven otherwise" then there would seem to be an issue. I would then simultaneously be saying that other _people_ aren't sentient (I can't prove they are) and that other _people_ don't understand things (I don't even know how to measure such a thing, after all).
> If you can't, does that mean that atheists, like me, can't continue to say very confidently indeed that he doesn't?
I'm not quite sure I follow. Indeed I hold that strong claims that God either does or does not exist are unreasonable on the basis that both would appear to be fundamentally untestable claims. At least, untestable short of such an entity choosing to provide incontrovertible proof of its existence.
> Yann Lecun is very salty about chatgpt, I wouldn't take his word seriously. With all due respect, he's not salty at all. He's even overseen plenty of cutting edge research in the LLM space. But he rightfully has pointed out what they can and can't do.
There's too many people encountering a chatbot for the first time that sounds coherent and engaging in anthropomorphism.