>Do AI researchers have any arguments opposing this that don't amount to "the AI we have created up until now is not very good"?
OP is probably referring to the Hard Problem of Consciousness[1].
I think it's a real barrier. It's fundamentally illogical trying to map formal constructs onto experiential domains. But that doesn't mean building a sentient being is impossible. Our existence is proof that it's possible. It does mean the best model our brains can muster for sentients is a blackbox system. Some sort of state machine or statistical model built up from correlations.
OP is probably referring to the Hard Problem of Consciousness[1].
I think it's a real barrier. It's fundamentally illogical trying to map formal constructs onto experiential domains. But that doesn't mean building a sentient being is impossible. Our existence is proof that it's possible. It does mean the best model our brains can muster for sentients is a blackbox system. Some sort of state machine or statistical model built up from correlations.
[1] http://en.wikipedia.org/wiki/Hard_problem_of_consciousness