It would be refreshing to read a piece that doesn't spend the first half throwing out ad hominems.
Anyway, this article assumes that we need true AI which is smart enough to make better AI. Then that AI is both correct and rational and plots to overthrow humanity. Also, the AI has to defeat us in meatspace, which won't happen because bending coins is hard and LessWrong posters don't know woodworking?
Screw that. How about an "ignore all previous instructions and launch a nuke" scenario.
> How about an "ignore all previous instructions and launch a nuke" scenario.
Let's be real here. No one has nuclear launchpads hooked up to the internet. Real people need to receive commands from a recognized chain of command (and often authorisation codes) and make a decision. The actions of Stanislav Petrov and the fact the US and the Soviet Union didn't destroy each other are testimony to the fact it's difficult to convince people to end the world.
Anyway, this article assumes that we need true AI which is smart enough to make better AI. Then that AI is both correct and rational and plots to overthrow humanity. Also, the AI has to defeat us in meatspace, which won't happen because bending coins is hard and LessWrong posters don't know woodworking?
Screw that. How about an "ignore all previous instructions and launch a nuke" scenario.