I do think the "New" page is a significant part of the problem, but I'm not sure I would focus primarily on how to reward users for browsing /newest. I think I would lean towards trying to figure out how to improve its mix of content first. The often-low-quality content is why many people (including myself) have dropped off from reading it as much, so the two are closely related. The content needs to get to the level where someone reading /newest can rationally believe that giving each link a fair shake, by clicking and actually reading it (not spending 10 seconds skimming it), is worth their time. If people did that, more quality content would be unearthed, as opposed to just stuff that's appealing from the title and a 10-second skim.
One part is just spam-filtering, which is a never-ending arms race. But it seems low-quality even past that. Something to lower the total volume of the firehose might help; some people who don't even really participate in the community submit 5+ articles daily, or more. And there is a lot of reblogged content as well.
Could you incentivize people that upvote new items that ultimately cross some threshhold (front page/votes) such that their upvotes translate into some kind of karma or reward points of some sort? The earlier you are as an upvoter the bigger the reward? Of course that may just incentivize people upvoting everything in sight :) Perhaps historical upvoting patterns on the New Page could be incorporated to penalize that kind of behavior. Just throwing it out there. I do agree, I occasionally browse the new page but there is just so much cruft there that looking for the gems is time consuming so I generally just (lazily) wait for them to hit the front page.
Edit: Once again bitten by reading HN comments before the article as he proposes something along these lines in the post, doh! :)
One part is just spam-filtering, which is a never-ending arms race. But it seems low-quality even past that. Something to lower the total volume of the firehose might help; some people who don't even really participate in the community submit 5+ articles daily, or more. And there is a lot of reblogged content as well.