The Nexus 4 was massively supply constrained, not available in most countries (either for a long time after launch or at all depending on country), didn't do LTE, and wasn't marketed heavily. If you're comparing it to the S4, which is the most marketed consumer tech product on the market and is available everywhere, it's not surprising it didn't sell.
It was never available in the areas that people wanted them most. Hobbling sales was more of a decision by Google than a lack of demand. I suppose Google just wanted to show that a device could be built at this cost and prove a point. Here in India the demand for the Nexus 4 is HUGE however, after taxes, import duties and markups, it retails for around the same price as a iPhone 4s which a lot of people prefer as a status symbol, never mind capability. Most discerning techies however, carry a nexus.
