The logic behind why it is done like that I get. Just wondering as You said is it possible to push at least the most bug-prone and exploitable ones to user-space
I don't see how you can convert a kernel-space driver to a user-space one without significant rewriting, and in some cases it may not be possible at all.
What about some abstraction/interfacing layer/driver that would take care of exposing some kernel functionality an average driver needs and provide additional validation?
The "most bug-prone an exploitable" dimension is a bad one. There's probably some correlation, but it is not a good way to look at the differences.
You can push into userspace the software that work some data into some low level data. You can't push into userspace the IO of that low level data to the hardware. If your driver is mostly interpreting complex data before IO, you can push most of it into userspace, but if it is really doing IO (or calculations are interspersed with IO), you can't.