The cross-race effect is a thing [https://en.wikipedia.org/wiki/Cross-race_effect] and is not unique to whites making this mistake, so it's a little bothersome that someone would have a problem if you get mistaken for someone else. I think it can be an honest mistake that anyone can make.