Unfortunately I had the opposite experience. As a Chinese guy in the rural parts of Pennsylvania I was stared at a lot but felt fine when I was alone.

But when I was with my white wife, I noticed a lot more sneers and menacing looks.