It's about the cost: 2GB RAM, 2-core 2GHz ARM CPU, GPU, 160GB hard disk, for just 100 US$. Add another 100 US$ for a case, keyboard, mouse and a cheap monitor... the whole desktop computer for just 200 US$ (!)
Obviously. But people need performance on the desktop. With cheap AMD / Intel QC CPUs do you think ARM will offer better performance at significantly lower cost than AMD/Intel? I doubt it. And just cost is not going to play well in the desktop market.
EDIT : At that configuration, without a monitor, when desktop sales are all time low compared to laptops in the PC market - why wouldn't one just buy a AMD Fusion Netbook?
Desktop performance, most of the time, isn't CPU-bound. HDD speed is the usual suspect.
These NVidia chips will most likely ship with custom circuitry to accelerate the few CPU hungry operations performed at the desktop (video codecs and 3D gfx). Keep in mind that at that point, at least 8 cores per chip will probably be the norm. Single core performance will not be that important.
Edit: of course, add CUDA to the mix regarding computationally heavy tasks without dedicated acceleration.