Intelligibility heavily depends on what you expect to hear, and that depends on your native language or even locality. Even a tiny amount of French accent in English makes it sound like gibberish to me (but not others, and I don't have this issue with other thick accents). I'm sure my native accent is also incompatible with someone else's ears. That's the reason people pay accent coaches.
I understand the feeling, but I can usually adapt pretty quickly when someone speaks in a very unexpected accent. Sometimes I can even remember the sounds well enough that I can reinterpret it on the fly without having to ask them to repeat.
