Exactly right. I once timed my departure from such a job within a period during which lines lost to attrition or retirement would not be replaced. This was soon after a reorganization. The loss of direct reports hurt the manager's street cred.