I've also been listening while walking around/using public transport for many years, you can usually just pause the thing when you're in a situation that requires more attention (e.g. crossing) but generally it's better than walking while reading.
The problem is more that if many things are happening you tend to lose track of the book.
But that's why apps have "go back 30 second" shortcuts.
I live in a suburb, we don't have many sidewalks and we don't walk anywhere. I've been yelled at and nearly run over for using crosswalks, and walking to get where you're going is something only the poorest of the poor do here. Sad, really.
Have you had problems being disconnected from the world around you with such a setup?
I'd like to take more walks instead of sitting more when I get home, but I'd be worried about being hit by a car or something I didn't hear.