Feeling safe is just as important as actually being safe.

I think you partially addressed that already with the privilege part. I know my city has rough areas that definitely have an increased risk of getting mugged, robbed or just jumped for no reason. Knowing that is enough to keep me from going around there unless I have a reason.