I've been working on a "newsbetting" site. Basically, you get a de-titled article and have to place a bet whether you think the source has a right leaning bias or left leaning bias. The idea is this betting market will force people to contend with their biases which will then reduce the proliferation of fake news. Theoretically
At the moment I'm using allsides.com[1]. There will always be some dispute about how biased a news organization is, but most people have the general understanding that, for example, Breitbart is more right than the HuffPo. At some point I'd like to add a voting feature, so you're basically trying to compete against the crowd
So really you're betting on what "allsides.com" thinks, not if an article is actually right or left leaning (since there is no objective measure of right/left leaning).
Somewhat. Allsides has a whole system[1] as to how they assign bias which is fairly robust, in my opinion. You're not betting on what the moderator at allsides think a source bias is, you're betting on what all of the patrons at allsides think the source bias is which is a little better.
But you are right in that no one can "objectively" measure partisanship since there is no such thing.