Seconded; it's a cost and politics issue to have open floor plans, and mostly to "put those uppity programmers in their place." Wouldn't want them asking for what they're worth, now would we? And hey, by having them constantly distracted and not able to concentrate, they won't be as productive and therefore not feel justified in asking for what they're worth! I mean, if open floor plans and cubicles were really that conducive to thinking work, you'd see it in many more places, such as research, doctors offices, law firms, etc, etc.
I don't think it's a "conspiracy theory" in the traditional sense (although I do admit that my post was slanted a bit that way). I'm fairly certain it is as Crito said, "unplanned [and] unorchestrated". And there's no need for conspiracy: simple human nature in the aspects of politics and pettiness can easily explain it - it doesn't matter if you can prove what is more productive or cost effective, management ("owners" or their representatives) will continually push for more leverage and control over their workers, until they experience pushback, profits be damned.
A company wouldn't. I posit that individual middle managers are the problem: whilst they do want the people working for them to be productive, they often subconsciously act to make sure they have the supposed "status" and that this comes (unknowingly, to them) at a cost.