This is a strategy, but it typically falls apart against clever attackers who are targeting you specifically. Hackers have been performing return-to-libc attacks forever where they don't actually get to write any code at all, just sequence code that already exists in your binary.

Java also tried this in a slightly more rigorous manner with the SecurityManager and that just ended up being a botch.