100% this, you've described perfectly how I feel. It's not something I have control over either. Even in high school, I've had to close the doors to my room to be able to focus on learning. I can work in a team on a common task requiring constant interaction (e.g. brainstorming, negotiating, designing), but if a task requires individual focus (like programming), I have to expend significant emotional energy on managing my anxiety if my work area is in the open.

I know I'm not alone in this; the last time I worked in an office, we were technically in a semi-open space (8 people in a room), but the desks and monitors got subtly but quickly rearranged to block everyone's line-of-sight to everyone else.