I believe in many places (Western countries) white people are more easily scared of black people (big or not) than big and white. I guess that's what the author was talking about.

I was born I Brazil and now live in UK where, despite the abyssal difference in physical/verbal violence against black people, it's still very easy to see how white people act more "jumpy" around black people. Mind you I'm "very" white (german descendant) and even to me it's clear the difference.