A family generally supports each other and works together. Parents help the children, etc.
Knowing this, wouldn't it make more sense to see if our society follows a meritocracy on the family level than on the individual level? Individually our society might not follow a strong meritocracy, but familially we might.
Now the question exists, is that still a meritocracy? On the family level, yes.
Are we okay with giving up a meritocracy on the individual level to allow a family to help each other? Probably yes to that as well. We should be more concerned with family meritocracy than individual meritocracy IMO.
See the British upper class for a fine example of wealthy inheritors void of merit.