By the time a single AI system can prove those theorems and learn to perform well on very unrelated tasks, it could be too late to think about AI Safety.
AI Safety researchers and I are not arguing that AGI will certainly arrive in a specified amount of time, it just that we can’t be sure.
There are a significant number of AI researchers, however, who believe it might get developed within a few decades.
OpenAI, for example, is aiming for it; DeepMind as well. Both have research programs on AI Safety.
Do you have any other objections to the reasoning in the OP (no fire alarm..)? Subjective implausibility is not a good one. Your argument rests on using only “existing techniques”. When thousands of brilliant minds are working in the field and hundreds of good papers are being published every year, how can we be sure there won’t be a novel technique that can perform outside current limitations within a few decades?
At least two top computer vision researchers I talked with a couple years ago said they didn’t believe what their groups could do just 5 years before they did it.
AI Safety researchers and I are not arguing that AGI will certainly arrive in a specified amount of time, it just that we can’t be sure.
There are a significant number of AI researchers, however, who believe it might get developed within a few decades.
OpenAI, for example, is aiming for it; DeepMind as well. Both have research programs on AI Safety.
Do you have any other objections to the reasoning in the OP (no fire alarm..)? Subjective implausibility is not a good one. Your argument rests on using only “existing techniques”. When thousands of brilliant minds are working in the field and hundreds of good papers are being published every year, how can we be sure there won’t be a novel technique that can perform outside current limitations within a few decades?
At least two top computer vision researchers I talked with a couple years ago said they didn’t believe what their groups could do just 5 years before they did it.
See this thread for more on the rationale for preparation: https://news.ycombinator.com/item?id=23414197