Yes, ultimately each is responsible to himself/herself. But it doesn't help if someone higher up is taking advantage of such people, knowingly. It is nearly the same as taking advantage of people with low self esteem etc. The managers and higher ups needn't help, but at least they can stay away from making the situation worse.