Because otherwise anything can be of arbitrarily large dimension, and thus the whole term loses meaning.

Your intuition is in the right direction, but not quite correct.

Think of a cartesian plot. Now, think of the Y axis "tilted" to the front. With those 2 "vectors" you can still describe the whole plane (for whatever "describe" means =P) If you tilt it so much the "Y" axis becomes paralel to the X axis, then you lost something.

Instead of "orthogonal" what you need is a mathematical property called independence (as in linearly independent) that basically means "not redundant to express a space"