Does that take any enjoyment out of reading for you? I tried speed reading for a brief stint, but then decided to stop. For me reading a good book is like savoring good food, I wouldn't want to chug it down.
Not speaking for btilly here, strictly for myself: I think it depends on the kind of book, kind of writing, and personal expectations. When reading a novel, I'm mostly interested in the story and character development, and less in things like long descriptions of the environment (which Jean Auel does, a lot, though maybe not as much in the first one IIRC). I'm like the kid always asking "what's next" even before the narrator finishes his sentence.
So I like to read fast, and maybe there are some things I'm missing but I don't feel like that's a loss.
Mind you I'm not really speed reading (I think), I just read pretty fast. And skip uninteresting pieces of text (by skip I mean read some words or parts of sentences here and there, like the start and/or end of paragraphs, to know what the text is about so I know where to resume normal reading).
Now food is something else entirely :) I'm a pretty fast reader but a slow eater.
It depends on the book. Most sci-fi/fantasy I enjoy at speed. Anything where I want to find out what happens NEXT. But poetry and math I would want to read more slowly. I read more quickly than I think, and a full appreciation of the rhythm of language takes time.
What I do isn't what most people call speed reading, though. I don't skip any words, and can recognize and pull out exact phrases. I don't know how much text I could hold like that. When I was 19, a long book. But I think my capacity has gone down with age.