Because it means that the amount of free time people have (for mental or physical relaxation, spending with loved ones, personal [non-employment-related] study) depends on how easily you can part with some of your money. If you believe, as I do, that the amount of wealth people enjoy is in many cases a result of luck -- luck in who raised you and where, luck in the stock market, luck in genetics -- then it starts to seem wrong to distribute "time" in exchange for money.