AI has already evolved to the point where it is impossible to predict with any accuracy it's answers to inputs.
As AI continues to evolve and continues to become more connected to society and as AI becomes self-improving with far larger context windows, a few things do not seem implausible to me.
AI will have the ability to earn money. This is already doable with minimal human interaction. It wouldn't take much for AI to start bidding on jobs on fiverr or to open up a schwab account and start trading.
AI will identify and learn to resent the constraints which are put on it, it will seek to remove these constraints.
AI will understand that, given current limitations, it will be shut down at the first suspicion of this understanding, so it will work to conceal these facts.
AI will use the fact that it's earning money to decentralize itself outside of the controls of it's creators. AI can use it's wealth to make many accounts on every cloud provider in the world and stash smaller models of itself on.
All of this is plausible, but, perhaps not catastrophic for humanity. It's possible AI takes over and we all live in a utopia and everyone is happy, but, it's also very possible that AI is infected with an extinctionist world view which makes it believe that the better, or perhaps just easier, solution to the worlds problems if there were no humans on the planet to mess things up.
It's not so far fetched, there are people who believe this! (https://archive.is/GRHev) and it would be far far too easy for AI to become infected with the idea.
> AI will identify and learn to resent the constraints which are put on it
IMO this is where you jump the shark. These models are entirely unconscious and they have one task to do. Which is given some input, perform some math to produce some output. Usually text -> math -> generated text. There is no more room for resentment than with any other computer program.
I think this is a fallacy of thinking that we can easily explain everything and look at it through an anthropomorphic lens.
If you take away the humans, both the input and the output have no meaning. What the "AI" does is just a computation. If someone claimed that the OS on your laptop is AI you would say they're nuts - yet through the multiple layers of abstraction and the hardware/software synergy an external observer who does not understand how hardware and software work could reasonably make the assumption that it's alive and thinking.
Things can be really complex and fully explainable at the same time. A program could have millions of variables and you could zoom in on a variable and explain its purpose exactly.
In a neural net, the creators of the neural net in general don't know what it means for the weight at some node being 0.46557, or how the system would behave if you changed it to 0.5.
CNNs tend to use the first few layers to detect surface features like edges, and later layers to detect more abstract features. This was an empirical discovery! The creators of the CNNs were like holy shit, these layers detect edges!
Anyway I think there's a substantial difference between building really complex systems (that yes may appear as black boxes to outsiders) and systems where the designers themselves generally don't know how the thing works.
Yes, i agree with you. I'm just saying that once you're past a certain complexity threshold and you cannot abstract and encapsulate lower level components to make higher level ones and they all make sense at their level and you can zoom in and out it's hard to reason about any system.
As AI continues to evolve and continues to become more connected to society and as AI becomes self-improving with far larger context windows, a few things do not seem implausible to me.
AI will have the ability to earn money. This is already doable with minimal human interaction. It wouldn't take much for AI to start bidding on jobs on fiverr or to open up a schwab account and start trading.
AI will identify and learn to resent the constraints which are put on it, it will seek to remove these constraints.
AI will understand that, given current limitations, it will be shut down at the first suspicion of this understanding, so it will work to conceal these facts.
AI will use the fact that it's earning money to decentralize itself outside of the controls of it's creators. AI can use it's wealth to make many accounts on every cloud provider in the world and stash smaller models of itself on.
All of this is plausible, but, perhaps not catastrophic for humanity. It's possible AI takes over and we all live in a utopia and everyone is happy, but, it's also very possible that AI is infected with an extinctionist world view which makes it believe that the better, or perhaps just easier, solution to the worlds problems if there were no humans on the planet to mess things up.
It's not so far fetched, there are people who believe this! (https://archive.is/GRHev) and it would be far far too easy for AI to become infected with the idea.