There is a similar "karma" system on my social network, and one of my users suggested that we limit the amount of karma a user can receive within a give day. On YC, it makes sense to limit the submission volume according to the person's karma level. Someone with high karma is unlikely to be spamming the site.
This, combined with a burying system to get rid of junk stories, could clean up the 'new' section substantially. It would also be useful to factor snr into the computation; the nature of the karma system, which makes it more advantageous (at a marginal level) to submit a story than a comment, encourages a shot-gun approach to submission; SNR would combat that.
EDIT: Ha, points to jwecker for posting about SNR while I was writing about it.
I've never used Digg, but from what I read of them, their burying system is working out rather poorly. Burying a story on Digg requires an O(1) number of votes, so anything controversial gets buried even if 90% of users agree that it's an excellent article. A simple up/down model like Reddit's seems like a better idea, but see the top-level comment that I'll have posted in about two minutes.
Well, there's also signal-to-noise ratios. There are some people who have high karma by submitting dozens and dozens of sites knowing that one or two has got to stick. To me that degrades the quality of the site as much as spam does. I guess your solution might solve that in a round-about way as well.
I would abstain from complicating the user experience, though. In my experience, posting a topic has much more potential in gaining points than writing *thoughtful* comments. A good strategy to rise above that ranks is to post a bunch of interesting links; upon achieving high points, the user can then take a defensive stance by defending his or her opinions through posting comments instead.
The point is, from a user's perspective, there is more to gain from posting topics than comments.
That is certainly the status quo; I would ask, however, whether it is desirable. I think the system should provide more incentive for thoughtful comments, or at least some sort of disincentive for shotgun-style submissions. It is difficult to build karma by posting lots of comments; you have to put the effort into generating good content to get karma. The same should ideally be true for submissions; the difficulty, of course, is that since the content of submissions is generated by someone other than the submitter, simply posting lots of them /will/ eventually result in a karma-boosting hit.
We had a similar problem on our social network where users abstained from giving karma to thoughtful comments. Consequently, we coded an algo to add "+0.1 karma" to every user that gave another person "+1 karma". That was enough to take care of the problem.
I think there are a couple reasonable assumptions about high-karma users that we can work to our advantage:
1. They tend to spend more time reading the "new" queue than average, because they spend enough time on the site to exhaust "top" and want to read more.
2. They're more likely than average to be open-minded about controversial stories as well as less tolerant of inane ones. If they spend more time on the site, then it's more to their advantage to promote interesting stuff rather than promote an agenda. It's dumb to piss in your own pond.
Here's what I propose: high-karma users get more voting power over *young* stories, to enable them to police the "new" queue and out-muscle spammers and voting rings. After stories are a few hours old, high-karma users no longer get a boost if they haven't voted on them yet. If high-karma users have already voted on a story when it reaches the aging threshold, the boost goes away for upvotes but not for downvotes. That way:
1. Bad stories never see the light of day, because high-karma users bury them and they stay buried.
2. Controversial stories will make the front page, but they don't stay there long if the rest of the community doesn't like them.
3. Good stories make it to the front page slightly faster.
Dang, already? Sounds like Arc is working out well.
Still, though, any statstical approach (which is what I assume you're using) is going to have some lag because you have to wait for a statistically-significant sample. You can catch any given offender after he's caused an O(1) amount of trouble, but if the offenders don't correlate with each other, you can get overwhelmed by a sufficiently large number of them. Spammers certainly do correlate with each other, but it's not obvious that voting rings do.
I'm prone to think that pg's algorithm is, to some extent, content based. I have no real evidence to support this, just the evangelism in "A Plan for Spam."
Right. That's what I meant by "spammers correlate with each other". I'd be very surprised if content-based filtering were useful for catching voting rings. You can use graph-theoretic metrics to catch them, but unlike spammers, identifying one doesn't obviously help you identify others.
The word karma in the context of this site has two meanings to me- one is that if you give a little of yourself and write a thoughtful comment or needed submission then you'll be rewarded. The other meaning to me is this- give a lot of it out and it'll come back to you. So if you care about karma, be liberal with it. (and I'm not talking about moderating this comment up).
If there is a constant flow of karma, it's much easier to see a differentiation in comments. There's nothing more irritating to me than seeing a comments board full of great thoughts and lots of responses where everything is either 1 or 2 points.
[upd] - and, of course, the old slashdot rule- focus on modding up- have a very good reason to mod down. Sure wish the digg community had followed that rule (back when I still used it).
I like how you put it because I've always thought of "karma"-type points in social networks as money -- social currency. We want to stimulate the "economy" by having an active exchange of karma through the community, encouraging more and more discussions and contributions.
python_kiss's comment about limiting karma is also an interesting thought. Inflation is something to ponder about in this analogy.
There is no absolute Right supply of either fiat money or karma. If every dollar bill in everyone's wallet magically turned into two, the price of everything would double and nobody would be materially affected. Similarly, doubling everyone's karma and causing the up arrow to give two points rather than one would have no material effect.
Inflation matters when you factor in time. Inflation encourages risk. It makes your current savings become less valuable in the future, encouraging you to try to increase your wealth to compensate. Too much inflation encourages too much risk, and you get a bubble.
Money is valuable because it can be exchanged for scarce resources. Karma is valuable because it gets you attention, which is a scarce resource. There are two kinds of attention: visibility of articles and visibility on the leader board. Karma is the cause of the latter, but only a byproduct of the former. There are a fixed number of articles on the front page and a fixed number of spots on the leader board, but both become more valuable as the community grows, because more eyeballs means more attention.
At this point, the money analogy breaks down. Money is both a unit of accounting and a medium of exchange. Karma is only a unit of accounting. It tells us how wealthy we are, but we can't exchange it for other wealth. Attention doesn't behave like a commodity either. If we give and receive the same amount of attention, that's not the same as simply doing nothing.
So, news.yc really doesn't behave anything like a conventional economy. If any analogy can be drawn at all, I'd say that we're in a state of deflation. As long as the community continues to grow, producing articles now is less profitable than producing articles later.
One thing I find interesting about news.yc is the relatively small set of active contributors to the comments, and their rather high availability: case in point, this thread, which is feeling to me rather more like an ongoing conversation than a /. style shout-fest.
Anything that can be done to encourage this feeling of intellectual discourse is a win, in my opinion.
We're actively trying to avoid the nastiness that seems to take over so many online discussions. That's why there are fewer down-arrows here, for example. I suspect that down arrows are more often clicked on stupidly than up arrows, that if you don't have down arrows you give people fewer ways to inject stupidity into the system.
I'll probably never get around to supporting bold text in comments either, for the same reason.
To some extent news.yc is protected by being about a topic only a small number of comparatively smart people care about. With any luck we'll never have the full-blown trolls you find on general news sites. I have some ideas for solutions if trolls do start to appear.
I've joined well over 50 social networks and what I've noitced is that freedom to choose appeals more to people than the right choices already made for them. More features, more niches, more startups, more choice. Users might complain about features, but fundamentally they like choice.
The more time a user spends making choices on a network, the more loyal he or she becomes. You don't spend 12 hours of your life customizing your profile if you don't plan on sticking to it.
Lastly, too much of anything is bad. The state should have control over medical supplies, education and energy. But beyond that, it should trust the consumer with running the country.
I think you're doing the right thing by retaining the down arrow on comments, however. When someone writes a comment you dislike, there are four things you can do with it:
1. Ignore it
2. Downmod it
3. Write a quick reply
4. Write a thoughtful reply
Option 4 is ideal, of course, if you have the time, but you usually don't. Quick replies are what we want to avoid. That's how flamewars start. Ignoring the post is going to be unsatisfactory for the flamewar-prone. That leaves the down-arrow as a sort of relief valve. So I think its presence can actually reduce nastiness.
Aha! I'd not had the realization that the the downmod arrow only appears probabilistically. It meshes interestingly with a notion that dfranke and I were discussing earlier tonight, namely the appropriate ratio of up to down mods. By controlling the prevalence of options to downmod, you can exert some control over that ration--very sly. :)
I love the bareness of this site so far. Keeping it simple may hopefully maximise the honeymoon period before the deluge of crap that any small web success appears to attract.
I suspect the ability to successfully transition from this honeymoon period to moderate popularity will become a defining battle for a certain class of social web start-ups.
Copy-paste from my reply to jwecker: "From a user's perspective, there is more to gain from posting topics than comments." Top users are more active in the discussion because they are already high above the ranks. The top user doesn't have to spam the site to gain karma, we already have enough.
This is an interesting notion; there is an implicit assumption that the low-karma user has something to lose by posting comments. Am I missing something here?
I do see your point, though, wrt top users not spamming the site. That's largely why I liked your idea of throttling submission rates based on karma. New users should have to develop karma by being useful, rather than just prolific.
Exactly, the user with low-karma is better off posting topics than comments. Comments don't receive nearly as much points as topics do. Even worse, comments require far more thought than blindly submitting links. What is the economic gain the user with low-karma receive from posting comments? Not much.
I know I suggested it, but throttling submission rates based on karma might not be fair to the end users. Once a karma-archy is established, the rich will get richer at the expense of the poor.
Well, I certainly hope that news.yc users are a bit more than Ricardo-programmed economic optimization automata. :)
One solution to older users having a perpetual advantage would be to implement some sort of aging of karma, or at least of the karma value used to determine throttling. Such a thing would actually be a win in terms of SNR--it doesn't really matter how karmic someone was a year ago, if they have only made low-quality posts/comments in the past month, they should be treated (in the submission system) as a low-karma user.
An addendum, stimulated by dinner with dfranke: under certain assumptions about the growth rate of the community, karma ages without any algorithmic intervention: as the community grown, the number of moderators grows, and so the mean and maximum moderation per post increases. As a result, new post are effectively weighted relative to old posts.
The problems, of course, are that there is some fixed point, and also that news.yc may not fit the growth model that makes the above true for a community like Reddit--and that may be a good thing.
Another idea I mentioned elsewhere- if a story gets to the bottom of the new page without any up-marks at all it adversely affects the submitter's karma.
I think this should remain neutral. Ignoring a submissions DOES indicate its worthiness to a point, but sometimes things get missed. Only a true down-mod should degrade the submitter's reputation.
I can see where we're heading with this idea. You're right -- to gain noteriety (karma), a user could easily engage in submission abuse. Submit enough links and enough will get upmodded.
Points, or karma, could be rewarded based off a ratio of useful contribution to total contributions instead. Such a system would encourage a user to make sure his contributions to the community are good. This would be beneficial implemented in any crowd-generated news site.
While not another news aggregator, our project is addressing similar types of issues in other channels. I am loving the applied relevancy to here already.
I automatically click on a submitter's name if I don't recognize it. If that submitter has only 1 karma point and is obviously new to the site, that submission gets extra special scrutiny- meaning if it is even remotely promotional I assume it's spam.
If you want to jump into the community here, esp. if you've already been lurking for a while- don't do it by doing a submission (unless it's a killer submission)- add some nice comments please!
Well I think we're at the former right now. Probably need a few more comments per post, but that will happen I think.
What we want to avoid at all costs is what happened at Reddit yesterday with the impeachment business. That's no good. I know they're working on it (spoke to them last night) but it was annoying there for a while.
Good point. On an unrelated note, however, your comment reminded me of another note I want to make: If your post is in response to a comment- please make sure and thread it. Because the site shows comments in the same nested level in different positions (either random or rotating or something, I haven't figured it out yet) each time the comments are loaded it gets real confusing if the comments aren't threaded correctly. There's plenty of horizontal space in the site's design so we shouldn't be reluctant to use it. The alternative, of course, is to quote what we're replying to.
I'm not a fan of threaded nature of comments. It's done to faciliate exactly that -- commenting, and does not provide for a proper discussion environment. It's difficult to follow a discussion from start to finish when popular comments are placed at the top.
Has anyone noticed that submissions tend to receive more points around 4 to 10 am GMT, Eastern Time? Posts that I make after 9 pm usually remain at "1 point" throughout the day. This probably has something to do with the Silicon Valley's working hours.
I can tell you that at least for me that's when I try to read a chunk of new submissions regardless of existing score. Once something has a couple of points it is much more likely to continue to grow if it is worthwhile, but it takes slogging through a lot of crap sometimes, it seems, to find that underrated 1-point submission- even though there are plenty of them in there.
I'll be honest, I've not been nearly so good about doing my time on the 'new' page. In your experience, are there any strong patterns in good submissions? What I'm getting at is basically: are there any views on the new submissions list that would make finding the under-rated ones substantively simpler?
That's a tough q. for me- I generally have to just open 3 or 4 of them and make a decision of whether or not to up-mod it then do it again sometime later. Two kind of related thoughts, though, for those who don't spend a lot of time working on the "new" page:
If you concentrate on the top of the new page and like a submission and mod it, it will get exposure near the top of the top page and get some real good input usually, so that's the best place to start- just look at the 3 or 4 submissions at the top of the page and spend a few minutes.
Nevertheless, there are some amazing submissions that because they don't get a point in their first hour of existence (because everyone is eating lunch or busy submitting their own etc.) end up moving clear down the list without anything. If you want some good karma (the real kind) spend time with the 4 or 5 at the bottom of the list giving belated deserved credit for some posters.
Cain, if you're looking for up to date startup news, it is better to simply subscribe to rss blogs. Here are a few good ones: TC, Mashable, GigaOm, ReadWriteWeb, OnStartups, PaulGraham, Startup Reviews, The Startup Journey, etc.
On YC, it sometimes helps to simply sift through a user's past posts and comments.
I was pondering this phenomenon earlier today; it would be interesting to do a quantitative analysis of post and moderation rates (this would require access to the backing database). I have noticed that there tend to be definite hot times, and that posts made outside of those hot times tend to never get modded.
What we need is more incentive for active discussion among topics instead of a flurry of ho-hum links. The community built around the "news" is the most compelling part of YC News.
Incidently, the project I'm working on relates to this in many ways.
High submission volume + no comments or hundreds of one-liners = might as well set up a news aggregator.