it's not just trust. a minority person (by which i mean to include all and only the less powerful in society, which has a heavy overlap with racial and ethnic minorities) can seem completely non-threatening, but still won't receive the kind of treatment he gets because the other party cannot muster the mental energe to do it as they finds it hard to imagine such treatment "leading to" anything.
As a fellow white guy, I know it's a lot easier for me to walk up to stranger and have them trust me than many minorities would experience.