The US Marines have been known to evacuate US citizens from foreign countries if war breaks out. Think of your taxes as an insurance premium for global personal safety.
Country's passport has a value in itself. The one from the US enables you to leverage international travel treaties and visa arrangements. Just answering your question, not making a point on whether taxation is the correct way for US to "cover" for those benefits.
Well, in the US, one benefit is being able to freely travel to many other countries without needing to go through a months long visa application process.
Your social security benefits are derived, in part, from how much you paid in on your top earning years. Thus, if you didn't pay in you would have very very very little social security (which is already too small of a benefit to live off of). Thus, I don't think there is a moral hazard of social security that would cause people to return to the US for those benefits. They could amend the law (not sure if it's specifics) that you have to pay in a minimum amount if you want to be paid out while living in another country. Simple fix to remove any poorly aligned incentive.
Medicare is much trickier as every American over 65 is entitled to it, but I'm not sure how that coverage works if you live outside the US.
Social Security is not tied to US citizenship. A foreigner who has worked in the US enough years (and paid into Social Security during those years) can move back home and receive Social Security benefits upon retirement.
