You could be a "kernel hacker" though, particularly if you're working on kernel-mode device drivers without support from the hardware vendor.

To me, a "hacker" is someone who is involved in finding ways to get systems to perform in a manner outside the intention of its design. E.g., hotrodders could be considered car hackers.