I think people are generally concerned with “badly-designed” TVs and conflate “smart TV” with “bloated TV”. If Apple made an actual TV you bet nobody would bat an eye. In fact many advocate using Apple TV alongside their offline TV.

I agree that smart TVs are nice to have, especially when they receive Airplay, YouTube, whatever straight from my phone. Dumb TVs are pretty useless on their own, I only use mine to consume internet content anyway.