I'm a native English speaker. I learned Spanish pretty fluently in Chile, interacting with native Spanish speakers from several countries, and my personal experience doesn't entirely match the map. Chilean and Argentine are definitely harder. I don't think I've met a Venezuelan or Panamanian to compare. Colombian was the clearest, most comprehensible accent of any I encountered. Easier than Mexican and Peruvian that are both marked as easy on that map.

I'd assume there to be regional variations like most countries, right? Hard to blanket one way or another.

For example, my wife learned English later in life and can understand neutral/midwestern American fine, but has tons of trouble with southern and northeastern regional accents.

I know Chilean accents get harder to understand the further south you go. More idioms and slang and more dropped letters in speech.