> Time and again, we see these systems producing errors ... and the response has been: “We just need more data.” but... you start to see forms of discrimination... in how they are built and trained to see the world.
Thank goodness this perspective is getting out there.
This is wholly false. Machines which analyse the world (ie., actual physical stuff, eg., people) in terms of statistical co-occurances within datasets cannot acquire the relevant understanding of the world.
Consider NLP. It is likely that an NLP system analysing volumes of work on minority political causes will associate minority identifiers (eg., "black") with negative terms ("oppressed", "hostile", "against", "antagonistic"), etc. And thereby introduce an association which is not present within the text.
This is because conceptual association is not statistical association. In such texts the conceptual association is "standing for", "opposing", "suffering from", "in need of". Not "likely to occur with".
There are entire fields sold on a false equivocation between conceptual and statistical association. This equivocation generates novel unethical systems.
AI systems are not mere symptoms of their data. They are unable, by design, to understand the data; and repeat it as-if it were a mere symptom of wordly co-occurance.
I don't know, inference from data is literally how all decisions are fundamentally made. Why wouldn't it be possible to create models that learn this particular pattern?
Here's where the words "data", "pattern", etc. become unhelpful.
Learning, as in what we do, is not learning associations in datasets.
It is learning "associations" between: our body state and the world as we act. It's a sort of: (action, world, body, sensation, prior conceptualisation, ...) association. (Even then, our bodies grow and this is really not a formal process.)
This is, at least, what is necessary to understand what words mean. Words are just tools that we use to coordinate with each other in a shared (physical) world. You really have to be here, with us, to understand them. Words mean what we do with them.
Meaning has a "useful side-effect". It turns out when we are using words their sequencing reveals, on average, some commonalities in their use. Eg., when asking "Can you pass me the salt?" I may go on to ask, "and now the pepper". And thus there is a statistical association between the terms "salt" and "pepper".
But a machine processing only those associations is completely unaware there is anything "salt" to pass, or even that there are objects in the world, or people, or anything. Really, the machine has no connection between its interior and the world, the very connection we have when we use words.
When a machine generates the text "pass me the salt" it doesnt mean it. It cannot. There is no salt it's talking about. It doesnt even know what salt is.
A machine used to make decisions concerning people, unaware of what a person even is, produces new unethical forms of action. Not merely just "being racist because the data is".
We also tend to have decades of experience interacting with other humans and understanding what would be reasonable to want / moral / what would “make sense”.
This is a big part of why I’m bearish on things like fully autonomous self-driving cars until general AI is achieved. Driving is fundamentally a social activity that you participate in with other humans, at least until we built out nation-wide autonomous-only lanes that only allow (through a gate or barrier) autonomous vehicles with self-driving engaged in a way that normal vehicles can’t “sneak in”…I’m not holding my breath. Maybe I’ll see it in my lifetime (30s), maybe not.
Perhaps religious cult territory too. "The Algorithm" [1] is already being used to tell fortunes -- which if it just stays as a few people having fun with horoscopes I don't really care, it seems harmless. But I could also totally picture some cryptocoin charlatan getting revelations about The Spiritual Algorithm or something and starts selling AI for getting into heaven.
---
[1] By "The Algorithm" I just mean the public colloquially reference to trending formulas used on TikTok etc, which I'm sure probably uses machine learning somewhere
I agree. I have the same reactions to these guys as I did to religious zealots when I was a teenager; and later to "transhumanism". Alarm, disbelief, hostility.
There is a new secular religion underneath this.
"Uploading consciousness", "Aren't we all just data systems?", "We've built a neural network with as many (meaningless) neurones as the brain!"
It's the same disgust and denial at being biological animals that all religions foster.
Thankfully here, I think, the religion has predictions. It predicts a self-driving car, a "fourth industrial revolution", and so on. When these fail to materialise, some sanity will return.
(Of course they have already failed, the question is when in the next 5-10yrs people will realise.)
> It's the same disgust and denial at being biological animals that all religions foster.
Huh, I didn't think of it that way! Good point!
I love history and a few weeks ago I read passage in a history book on a period in the early 19th century dominated by "American radicalism" (a liberal movement) and "transcendentalism" [1], when many hucksters and itinerant pastors profited off of the hype of industrialization. It immediately reminded me of the modern crypto / AI hype. In both cases, it actually is simply more smoke-and-mirrors to remove culpable human elements from a changing system --- by replacing it with "AI" and "crypto" (today), or the "invisible hand" and "radical self-reliance" (the 1820s-1830s US).
> A machine used to make decisions concerning people, unaware of what a person even is, produces new unethical forms of action. Not merely just "being racist because the data is".
I think you can create models that learn this particular pattern, but the models being told that oppression is a bad thing will determine that being black is a bad thing and from there that black people are bad.
I think it’s impossible for a human to read a mainstream body of minority political work and not come out with an association between black and oppressed. The entire dominant narrative is that all minority groups are oppressed. That association is definitely present in the text. Maybe it’s the case that we need to explicitly remove all negative associations for things like skin colour (potentially a hard problem in its own right) to generate more egalitarian text. But it’s not merely a matter of AI getting things wrong some negative associations are actually present in the text.
The association between "movie" and "tv" is "played on".
The association between "jellybean" and "apple" is "smells similar".
The association between "black" and "oppression" is "suffering".
The association in each case is NOT likely to co-occur.
AI is not detecting any kind of conceptual association. It is merely recording co-occurance. By interpreting co-occurance (statistical association) as meaningful, you are imparting a conceptual association to text it does not have.
"Statistical association" is just a pun on "conceptual association". Machines do not detect associations; they record frequencies.
> AI is not detecting any kind of conceptual association. It is merely recording co-occurance.
Can you explain the famous example of “king - man + woman = queen” through this perspective? Naively it does seem to extend beyond statistical representation as it seems some semantic association is preserved through the mapping of language onto a vector space.
Well, it's essentially a coincidence mixed in with a bit of superstition.
K - M has no semantics, its more like something you'd find in a teen magazine. "K - M" means as much, "Prince" or "Jesus Christ" or any of a number of words.
The type of association underlying word vectors is just expecting to co-occur.
So there are many cases we can enumerate where our expectation that Q occurs is modulated by M occurring. So if M hasn't occurred in some text, we expect Q rather than K.
(Equally, in biblical literature, we might expect JC to occur; and in disney films, "prince").
And many of these famous examples are just tricks. Very nearly all of the industry sells this technology based on a few cases which happen to turn out "as a lay audience would expect", and neglect to include the many many cases where they do not.
And to be clear, we should not expect "K - M" to be "Q" in anything other than a basically statistical sense, relative to some texts. "King - Man" isn't a semantic operation.
> This is because conceptual association is not statistical association. In such texts the conceptual association is "standing for", "opposing", "suffering from", "in need of". Not "likely to occur with".
The better GPT gets, the wronger you will probably be. Why a machine wouldn't be able to abstract conceptual associations from a statistical framework?
Conceptual associations are not in text. The frequency with which words co-occur says nothing about why they co-occur.
Deep Learning systems merely interpolate across all their training data: ie., they remember every since document they have been shown and merely generate a document which is close to a subset of their previous inputs.
This seems meaningful only because they've stored (a compressed representation of) billions of documents.
There is something, frankly, psychotic in thinking the text these systems generate is meaningful. Run GPT twice on the same input, and the text it generates across runs contradicts itself.
To think these systems are talking to you is to "read a telephone directory as-if it had hidden messages planted by the CIA".
GPT if it says, "I like new york" does not mean that. It hasn't been to new york, and doesnt know what new york is. It has no intention to communicate with you; it has no intentions. It has nothing it wants to say. It isn't saying anything.
It's a trick. An illusion. It's replying fragments of a history of people actually talking to each other. It's a fancy tape recorder. It has never been in the world those people were talking about, and when it repeats their words, it isn't using them to say anything.
None of what you say is incompatible with GPT being able to understand these concepts a few generations down the line.
I mean, your central point is that GPT could not possibly understand these concepts because it only perceived them from text, not real life, but... that's kind of true of people too?
I can make observations and guesses about New York, even though I've never been in the US in my life. I can try to understand the hardships faced by minorities, even though I have never suffered from race or gender-based discrimination.
A huge part of everything we know about the world around us comes from information we got from Wikipedia, or TV shows, or Youtube. It's information GPT could be trained on.
You can always make a philosophical argument that even GPT-6 won't "really" what it's saying, but we have yet to see what the upper bound of GPT is given enough computing power. I'd expect most non-philosophical predictions about it can't do to be falsified within a few years.
I am not sure if you are being deliberately obtuse or simply unfamiliar with how ML is designed and implemented. Almost every single point you mentioned does happen in practice. Most are limited by budget and scale, just like real world experiments.
Thank goodness this perspective is getting out there.
I have recently been incensed by the opposite view, that the bias is within "the data" only: https://www.youtube.com/watch?v=6jbin15-TcY .
This is wholly false. Machines which analyse the world (ie., actual physical stuff, eg., people) in terms of statistical co-occurances within datasets cannot acquire the relevant understanding of the world.
Consider NLP. It is likely that an NLP system analysing volumes of work on minority political causes will associate minority identifiers (eg., "black") with negative terms ("oppressed", "hostile", "against", "antagonistic"), etc. And thereby introduce an association which is not present within the text.
This is because conceptual association is not statistical association. In such texts the conceptual association is "standing for", "opposing", "suffering from", "in need of". Not "likely to occur with".
There are entire fields sold on a false equivocation between conceptual and statistical association. This equivocation generates novel unethical systems.
AI systems are not mere symptoms of their data. They are unable, by design, to understand the data; and repeat it as-if it were a mere symptom of wordly co-occurance.