If it comes up so often, then that is usually a pretty good indication that it's a topic that requires more discussion. Waving it away is, then, just going to make it come up again the next time.

For what it's worth, I do agree with Uptrenda. I do also want to explicitly highlight that this certainly is racism - and that whether it is justifiable racism is a separate discussion entirely.