I've wondered for a long time whether the following system would work:

Everyone can vote posts up or down if they like. There is a feature which statistically analyses how your votes correlate with other votes, and you can choose to see votes that the analysis predicts you will find interesting and worthwhile based on what you have liked seeing in the past.

Ugh. Count me out. I'm not a fan of how the trend online is towards algorithmic recommendation rather than on editorializing. I mean, I understand why a user-based moderation system is good. I like that content is pruned out a bit. At the same time, I think that public points mean ultimately ignoring complexities of a topic, thread, or person for the sake of easing consumption. I'm not a fan of that.

A recommendation system is even worse. That actively encourages people to look only at the path of least resistance for them. It means less emphasis on people who argue valid points that you don't want to hear and more emphasis on groupthink.

If anything, the trend should be reversed, and people should feel freer to give their opinion without a chance of whiplash.

I understand your reluctance, and I'd love to see strong editorializing on some site on the internet. It seems, though, as any site becomes reasonably large the editorializing disappears and is replaced with algorithms.

Has the idea I proposed (essentially the Netflix recommendation system) ever been tried before for a comment/story posting system? If not, why not go the whole hog and try it? Let's see if it works.

That's why I'd propose a forking. Create mini-HN communities and slowly thin out the userbase. That's if the site ever gets too big. It's perfectly manageable right now for me - I don't think new systems are necessary.

Ask yourself: has content recommendation ever worked? I've never seen an "If you liked X you'll like Y" work for any original content. Even for movies/books it's shady at best. Working with something as complex as comments is too tricky to make worse. How do you measure tone versus stance versus wit versus writing style? They're all elements of what I look for in comments.

Just because it's never worked as a movie recommendation system doesn't discourage me from trying it for a commenting system! (Especially since the idea is so trivial, and ought to be easy to implement).

Well, how would you display these recommended comments? How would you analyze the content of comments to determine recommendation? If I downvote every anti-Linux comment I see, does the algorithm stop showing me anti-Linux comments? Because that detracts from the debate going on. What if I downvote based on poor spellings - or even more, if I downvote solely based on how well the writer forms sentences? How will the algorithm figure that out?

The easiest solutions are the simplest ones. Complexity only serves a purpose when there's a simple cause behind the complexity. Adding algorithms when there's no need will only make things less reliable.

That is interesting... (you get my upvote :D)

You could take that farther and create a "pool" of good users who vote and comment in keeping with the HN style (a pool that could, of course, grow over time) and if your voting habits correlated to them your vote began to be worth more.

Indeed. Train on the Oranges, test on the Greys.

Thanks! I think you're right, that in order to avoid fragmenting the site into a lot of different subclasses with different tastes that there could be a pool of "good" users who determine what the default "good" story and "good" post are.

I think you're subconsciously reinventing k-means clustering with random starting points.

Subclasses might not be such a bad idea at all actually, after all there are different coffee shops for different tastes as well. If a site could be made to automatically appear as a place that caters to your 'taste' that would be a pretty awesome development. Everybody would feel right at home.

This is a fair point.

Some people are here for startup stuff. Some people for codign/programming stuff. Yet others (though probably a smaller demographic) are here for social stuff. And more.

Plus others are here for a mix.

Some way to tailor what you see would be awesome! not too much that each "genre" is different but enough that the posts or articles you want to see are at the top :D

I know very little about statistics so I wouldn't have much of a clue about the right algorithm to use, but I do have a good idea of the behaviour I would like.

I think the behaviour I want is exactly that required by the Netflix Prize:

http://en.wikipedia.org/wiki/Netflix_prize

although rather than rating comments 1 to 5 I think I'd probably just have three levels (don't appreciate, neutral, appreciate).

Using median instead of average would at least help the sensationalist voting problem.

VERY good point!