I think it could be easily attributed to the fact that most non-native speakers learn English at school by studying its grammar in written form, where the two words are distinct. Native speakers, instead, learn English as their spoken language, where the words sound basically identical to each other.

Yes, true. "have" and "of" don't sound same at all in almost all non-native accents which helps.

True. Non-native speakers learn the language a lot more "explicitly" so they make mistakes like these a lot less often.

Another pet peeves:

"irregardless" isn't a word. It's regardless.

If you don't care about something, you "couldn't care less about it". It's simple logic and yet people mess it up all the time.

I've only noticed it enter common usage recently, but I cringe whenever someone fails at using the word "devoid".

As a non-native, I learned "then" and "than" in different contexts, months if not years apart. I also learned them in speech and in writing at the same time.

Please don't quiz me on how to read bear, pear, tear, fear, spear, clear, and dear.