I think the idea of the U-2, which was developed before stealth was really a thing, is that although you can see it, it flies so high it is out of range of your interceptors.

After Gary Powers, they probably double-check that the countries they fly it over don't have modern AA capabilities.