No, you're not the only one. I, and at least three people I know, all found it to be an entertaining and fairly straight-forward read. Certainly not unfinishable.
I'd be interested to know what it is that people find makes them stop.
I found it light, and boring because of how light it was. I had studied Gödel's theorems, etc., so I really could not take the pace for explaining the logic stuff (most of the book) in GEB.
By the way, I would recommend any programmer interested in physics "The Road to Reality" by Penrose. I think that's a very challenging book, that can be interesting for programmers, even if it's not about programming. Compared to that kind of material, GEB is very, very light.
It may have been that I just read it at the wrong time in my life, but I found parts of it really tedious. I can imagine it would've been more enlightening if I'd read it earlier in my life, as opposed to trying to read it after I already had a CS degree and was in grad school studying AI...
I'd be interested to know what it is that people find makes them stop.