Hacker News new | past | comments | ask | show | jobs | submit login

A human might know the ultimate goal of sex is to reproduce, but still choose to use contraceptives. It's defeating the purpose of the original goal and just pursuing what feels good.

The same thing could possibly (likely?) happen with an AI system. While it might be able to reason about the intent of it's design (what the code's supposed to do), it'll still pursue what "feels good", which is an artifact of it's actual design (what the code actually does, bugs and all...).




I'm imagining a self-aware AI doing good all day then retiring to it's private realm to watch videos of paper-clip manufacturing.


This is one of those jokes too funny to keep to yourself and too contextual to tell other people.


Sex is how humans reproduce, but it's a pretty big (and highly metaphysical) stretch to say that that is the "ultimate goal" of sex. As far as we can tell, sex and reproduction have no ultimate goals at all--they just are. We have some idea of how they evolved over billions of year, but we have no idea how non-life became life, or why.

We're into some pretty deep stuff here. But the point is that no master creator has commanded us to reproduce infinitely and spare no expense. In fact our increasingly conscious control of our reproduction is reducing, not increasing, our risk of a catastrophic population explosion.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: