Maybe for text specifically you're right, but we use screens for other media like pictures and video. If we have to tone down contrast to make high-contrast text comfortable, we are also reducing contrast of other media, and this loss may be significant.
Now, if we had reflective monitors (like e-ink etc) we could just be comfortable with maximum contrast text (like printed text). I think the display technology is a problem here.
(But still: there are some books with yellowish pages which I find more comfortable to read than ultrawhite pages. Maybe such lower contrast pages are more expensive or something, or not adequate for all lightning conditions etc)