You can do the arithmetic any way you like, I suppose, but there is definitely a reason to always think of a matrix as a rectangle, and the article went to some length to explain why. A 2x2 is not 4 independent real numbers thrown into a box, it is a set of vectors that describe a linear space in relation to the "world", and it doesn't really make sense to describe a linear space any other way than by writing down the vectors that make up the axes of that space.
Just one of the many reasons you don't want to use R^4 to describe a matrix is because, for an orthonormal basis, A^-1 == A^T. The inverse and transpose are the same thing. That doesn't work in any arrangement except a rectangle.
This is what a 3x3 matrix is: https://dl.dropboxusercontent.com/u/364079/WhatAMatrixIs.png
Just one of the many reasons you don't want to use R^4 to describe a matrix is because, for an orthonormal basis, A^-1 == A^T. The inverse and transpose are the same thing. That doesn't work in any arrangement except a rectangle.
(edited for spelling)