I agree it's a big "if". For me, simply reducing the risk to less than the risk of the status quo is sufficient to count as a win.
I don't know the current chance of us wiping ourselves out in any given year, but I wouldn't be surprised if it's 1% with current technology; on the basis of that entirely arbitrary round number, an AI taking over that's got a 63% chance of killing us all in any given century is no worse than the status quo.
I agree it's a big "if". For me, simply reducing the risk to less than the risk of the status quo is sufficient to count as a win.
I don't know the current chance of us wiping ourselves out in any given year, but I wouldn't be surprised if it's 1% with current technology; on the basis of that entirely arbitrary round number, an AI taking over that's got a 63% chance of killing us all in any given century is no worse than the status quo.