You're not alone. Things are getting better, though. Drivers are definitely still an issue, but I think because of Steam's movement in the Linux market a lot of progress is going to be made... of course, maybe all this SteamOS stuff is just a hedge against Windows (to dissuade a move to a 100% app-store model on desktop).

Laptops and wireless are still definitely a big issue. I think desktops have a much better chance of working. Whether they work as well as Windows or not is a different question.