SMS is a much bigger issue if you are in another country, temporarily lose access to your phone, or work in a building without phone reception. It's prone to SIM swaps and could be intercepted by other apps on your phone. It requires having a fixed, serviced phone number, which people don't always have: children, homeless people and recent immigrants might struggle with that.

All of these scenarios happened to me, and I'm a fairly normal person.