This seems like a simple case of different people finding different things unintuitive.
My theory is that many people have a mental model of colors that maps roughly to the hue and value in HSV, so for example terms like "dark orange" is a value followed by a hue. Now suppose you understand brown as a "dark beige", that's suddenly confusing because beige introduces saturation and you cannot map beige back to orange without thinking about saturation.
> Now suppose you understand brown as a "dark beige"
I think my point is brown is dark beige, and beige is light brown. Neither is canonical. Is orange canonical? Maybe it's high frequency red? Or low frequency yellow? That's what Newton thought! It's all relative.
My theory is that many people have a mental model of colors that maps roughly to the hue and value in HSV, so for example terms like "dark orange" is a value followed by a hue. Now suppose you understand brown as a "dark beige", that's suddenly confusing because beige introduces saturation and you cannot map beige back to orange without thinking about saturation.