Principal Components is a wonderful concept, together with sister concepts eigenvalues/vectors, and orthogonality. i wish i could force everyone i talk to to internalize these ideas so that I could have more useful discussions with them.

that said, yeah not everything is linearly separable