"they don't necessarily deserve to have them when it comes at the expense of even a mild cost to lots of people."

That argument has been used to justify demolishing inner-city neighborhoods to make way for overhead freeways benefiting suburban commuters.