Speaking only to your first point, I don’t think the implication is that the white man holds a job that a better qualified non-white-man should have gotten, I think the implication is often times that the white man should not have gotten the job because it would have been preferable to hire an equally qualified non-white-man solely on the basis of his or her non-white-mannedness. This often makes people defensive (I have been as guilty of this as anyone else, as a white man) because it may seem that they’re being told they are “taking” a job from someone who deserves it more not based on merit or even tangential career skills like networking or interview prowess, but just because they have a different skin color and/or chromosomes.
Postulating "bias" in the hiring process does seem to imply that a non-deserving or less-qualified candidate was hired. Saying that being a non-white-male is itself a merit doesn't change that, and is also highly contentious.
