It's a primary reaction : animals dislike to show their back to "enemies", even if coworkers aren't technically enemies (some aren't really allies either).
I've noticed, when you invite a guest to sit in a room, the guest will almost always takes the chair with the back facing the wall.
Agreed, I intensely dislike having my back to a room (including in places such as restaurants), and this was always my explanation, too.
(Actually, the explanation I give to people is that I'm terrified of assassination attempts, which is related from a scientific point of view but has the bonus of making my life sound much more exciting than it is.)
I've noticed, when you invite a guest to sit in a room, the guest will almost always takes the chair with the back facing the wall.