Not true, modern BT codecs can be essentially transparent, and cheap 3.5mm jacks can have all sorts of issues. Bad noise performance, bad interchannel isolation, pop click issues, etc. don’t underestimate how badly you can design a 3.5mm jack especially if the phone only costs like $50 total.
As a non audiophile who designs this stuff for a living, you’re correct it’s not difficult or even particularly expensive. But it’s also surprisingly common to do it poorly.
It’s much easier to make a headphone output that is tailored to a very specific driver than it is to make a generic headphone driver that can drive any conceivable headphone plugged in.
Even a cheap phone’s DAC is going to sound better than any BT codecs.