This is very different from my experience. Whenever someone I was in a conversation with brought up GEB, it was always a great pleasure of mine. I'd get the chance to discuss the main ideas of the book, and the way I assimilated them. I tend to not even engage in conversations with people who do it mostly to show off the extent of their knowledge. I believe this second point is the important one. GEB is completely orthogonal to the problem you describe.