Not many. The biggest bummer is having to permanently reside. There is no way out or fudging it. Since all residents (including citizens) have to register their resident card (which ties your insurance, taxes, legal residence etc), there isn't much of a way out. (You can't be in two places of course! And you can forfeit your acquisition or penalized if discovered, although I haven't heard any such case yet.).

Think of zairyu (resident permit) as SSN with all personal details locked in.