I agree, and in fact I only added this feature because a client requested it ;-) but I'm glad it did because it can be fun if it's done right.
It can be very user-hostile, and reminds me of criticisms of Flash.
I obviously can't control what our users will do, but my own guiding principle is that the user should always feel in absolute control of the page. I've been to so many sites where there's essentially a small movie that you scrub through by scrolling, and it makes it impossible to navigate normally or jump to the content you want.
It's not really good for a general, informational website.