I feel that moderating a platform as massive as Facebook is essentially impossible.
Automating exact matches for known bad content is easy enough. But when trying to automate without using exact matches, it becomes impossible when nuance and context can make a huge difference in the meaning of a word or sentence. Automation can get it wrong, and when it does, users want the ability to force a human to review it and make a decision, but bad actors will abuse and overwhelm the system.
> I feel that moderating a platform as massive as Facebook is essentially impossible.
Not impossible. A few years ago, one report found that Facebook would need to double the number of moderators, expand fact-checking, and take a few other actions. Facebook won't do it because they'd have to divert a small portion of their $39 billion in yearly net income toward that goal.
Moderation is one of those things that basically cannot scale (properly) without human level AI or an absolute TON of resources. Any generalised system lacks too much context to do a good job, and any specialised one is too expensive/resource heavy. You can somewhat get away with it on something like Reddit where communities moderate themselves, but that brings its own problems for both the platform and the communities on it.
Moderation absolutely can scale, platforms just don't want to pay for it. For two reasons:
- Moderation is a 'cost center', which is MBAspeak for "thing that doesn't provide immediate returns disproportional to investment". For context, so is engineering (us). So instead of paying a reasonable amount to hire moderators, Facebook and other platforms will spend as little as possible and barely do anything. This mentality tends to be enforced early on in the growth phase where users are being added way faster than you can afford to add moderators, but remains even after sustainable revenues have been discovered and you have plenty of money to hire people with.
- Certain types of against-the-rules posts provide a benefit to the platform hosting them. Copyright infringement is an obvious example, but that has liability associated to it, so platforms will at least pretend to care. More subtle would be things like outrage bait and political misinformation. You can hook people for life with that shit. Why would you pay money to hire people to punish your best posters?
That last one dovetails with certain calls for "free speech" online. The thing is, while all the content people want removed is harmful to users, some of it is actually beneficial to the platform. Any institutional support for freedom of speech by social media companies is motivated not by a high-minded support for liberal values, but by the fact that it's an excuse to cut moderation budgets and publish more lurid garbage.
Moderation can't scale over populations around the world with entirely different attitudes and behaviors. Enforcing a global moderation consensus is about as effective as the UN.
The problem is also that standards for what's 'acceptable' discourse vary wildly between countries and regions and subcultures and other groups. This might theoretically be ignorable by saying "well, everything will be run based on what's acceptable among tech bros in Silicon Valley", but that might in turn annoy many governments in Europe and Australia and South America and Asia for either being too lax on the restrictions/moderation or too strict on it.
For example, there are certain words which are highly offensive in some regions, but completely fine in others. Some pictures and imagery which are banned or seen as offensive in some places but not others, and other such differences.
There's also often a lot of subculture specific terminology that might come across as offensive to those outside said culture if they don't know the context or situation too. We've seen that with certain tech related terms in the past.
Political misinformation and outrage bait is similarly not always simple to classify. Is content from a satire site that too many people take as real 'misleading'? I know there have been problems with things like The Onion, the Babylon Bee, Hard Drive, etc where people took their stories at face value. It also often falls into 'things the government/those in power don't want others questioning' too, especially when it comes to certain political events and controversies.
Let's not even get into talking about conflicts of interest and scenarios where business value and fairness clash, like when certain political figures say things that would probably get any other account suspended. What if the US president, UK prime minister, president of the EU or some country leader promotes information that's abusive, misleading or dangerous? Twitter had to deal with that problem back in 2016, and just ended up pretending he didn't exist until his term in office was over.
And then there's just context. Is an insulting message aimed at someone an attack by a rival or bully or troll? Or banter from one of their friends, like how many friend groups and families can make friendly jokes at each other's expense? On smaller communities this isn't much of a problem since the people there know each well enough to tell the difference between personal attacks and jokes, but is that going to be the case with a moderator here?
So moderation gets kinda tricky due to all the context needed to know whether a message is abusive or not. A well moderated large platform probably needs people in a variety of locations, from a variety of backgrounds, with some sort of way of getting a group consensus if any staff member is unsure.
Of course the other incentives you point out don't help much either. Google and co want to automate everything, so the idea of using humans to tell the difference between quality and spam/abuse is never even considered by them. Some things like outrage bait are definitely supported by the platforms for their addictiveness, and then we get situations where the owners themselves are horribly biased/trolls/whatever and are happy to allow abuse so long as its from 'their team'.
In the US, the legal reality of free speech applies only to actions the government might take to restrict it. Regardless of what is good policy, the legal reality at the moment is that private companies have no obligation to provide free speech to their users.
This is being challenged by Texas and Florida, which have passed laws to require a level of free speech on the platforms. The platforms argue that their editorial control on what they host is itself protected speech (in the same way that magazines deciding what articles to accept is).
The states, for their part, argue that the media companies are more analogous to common carriers, which can be subject to non-discrimination laws.
The case was just heard by the supreme court, so we should get a decision at some point.
> There is also a parallel legal reality - that people have the right to free speech.
The right to free speech only means you can't be prosecuted for what you say. It does not compel any private entity to distribute everything that you want to say.
> The right to free speech only means you can't be prosecuted for what you say.
It is a much broader protection against government retaliation than just "cannot be prosecuted", but it is a protection against the government, and it includes protection against being forced to endorse and relay others' speech. Compelled speech is the opposite of free speech.
I don't think they should be able to claim "safe harbor" protection (or am I getting that confused with common carrier?) and then get to use censorship however they like. The spirit of safe harbor is that "we're not responsible for what random people post on here". It doesn't seem logically consistent to claim this legal protection but then curtail the content to anyone's personal sensitivities.
Section 230(c)(2)(a) is pretty explicit in allowing moderation.
Just because they are not legally liable for what users post, Congress did not want to prevent platforms from going beyond their minimal legal requirements if they so chose.
The alternative us a law that says "It is unreasonable for us to expect you to perfectly police your users; but we will hold you liable for imperfectly doing so".
Just curious: do you believe in net neutrality? (I think that's the cause of my conflation with "common carrier"). But if censorship is a valid reason to deny service, why can't charter or AT&T say "we dont like what you're sending over our wires, so we're blocking or throttling you" ?
Is it merely the fact that encryption blocks the carrier from knowing what the line is used for? Or said another way, should AT&T have the right to terminate a user's service if they were certain a user was posting, say, white supremacy?
> There is also a parallel legal reality - that people have the right to free speech.
Here are go again.
Are we going to have to have the same argument that's already been on HN a million times about whether or not "free speech" means compelling a private company to let you use their platform for speech?
Hate speech is most definitely not an idea that's made up by corporations on the spot. It's an idea that's codified into law in many countries worldwide. In most of the world, free speech isn't a right with no strings attached whatsoever.
exactly. like when Yoel Roth (former safety lead at twitter) worked w/ FBI to ban conservative accounts, critical of Hunter Biden. No one cared, and Yoel and the FBI intuitively understood that.
Automating exact matches for known bad content is easy enough. But when trying to automate without using exact matches, it becomes impossible when nuance and context can make a huge difference in the meaning of a word or sentence. Automation can get it wrong, and when it does, users want the ability to force a human to review it and make a decision, but bad actors will abuse and overwhelm the system.