Well, but then, you arguable don't have a single, IP, but rather none at all.
Also, as you usually still have only one (non-)IP, the CPE still does NAT as well.
And in any case, NAT does not prevent inbound connections, not even CGN. CGN might prevent inbound connections from the public internet, but your ISP can still connect to your internal network if you don't have a stateful filter on your side.