Indeed they can, For basic framebuffer, but for doing 3d acceleration, to get many nice desktop effects people are used to you basically have to speak their proprietary protocol to the GPU. You can do a little better than framebuffer, like the "nv" Xorg driver, but it's still not really sufficient for composting, video playback and other things commonly expected from a desktop OS.
There's an effort to reverse engineer this called nouveau, they've done a surprisingly good job in a surprisingly short time, but it's still far from perfect.
> There's an effort to reverse engineer this called nouveau, they've done a surprisingly good job in a surprisingly short time, but it's still far from perfect.
Nvidia has been putting a lot of hurdles on their efforts since the Pascal GPUs. If you look and pre-Pascal GPU the performance of Nouveau is really impressive, but Pascal GPUs are crawling to go beyond a few frames per second. It's really sad.
There's an effort to reverse engineer this called nouveau, they've done a surprisingly good job in a surprisingly short time, but it's still far from perfect.