Given that this is a Chinese model, I’m genuinely curious if researchers have been evaluating risk that these models could be used for soft propaganda or similar purpose?
As others have reported, English and Chinese queries return different replies on topics that are not kosher in China.
What’s the risk that such models could be used for nefarious purposes by providing propaganda/biased/incorrect/… responses that on a cursory glance seem factual.
It’s a fair question, but one we should be asking about all models, perhaps especially our own. It’s of course easier to see the propaganda of foreign cultures, and this should be investigated, but let’s not let ourselves believe that a model is more likely to contain propaganda because it is Chinese. It will just contain propaganda that is easier for us to see as propaganda.
Noam Chomsky and Edward Herman wrote extensively about propaganda in democratic societies in their 1988 book Manufacturing Consent. A nice introductory excerpt is here, and the first two or three paragraphs are enough to begin to see the argument:
Put as briefly as possible: propaganda in totalitarian societies is simpler. They just use force to remove people who say the wrong things, and state media to broadcast the “right things”. In democratic societies, institutional power still wants to protect itself, and this is achieved through more complex means, but it is nonetheless still rather effective.
> It’s a fair question, but one we should be asking about all models, perhaps especially our own. It’s of course easier to see the propaganda of foreign cultures, and this should be investigated, but let’s not let ourselves believe that a model is more likely to contain propaganda because it is Chinese. It will just contain propaganda that is easier for us to see as propaganda.
Yes, but in this particular case I'm coming from a viewpoint where I view China as a hostile power. So, at the moment, my worry is about that.
In future, if US slips into authoritarianism, which TBH it might depending on the outcome of the next election, what you note would become a very real problem.
So, putting it differently and more neutrally, is there any research being done on evaluating a political, and other, bias in a model or is it just being all put in the bucket of hallucination?
The point of Chomsky’s work in this case is to show that authoritarianism does not make propaganda more or less likely, it just changes the means by which propaganda is created and reinforced. Chinese propaganda is easier to identify as a foreigner, but the propaganda of your home country has a much more significant effect on your life. The nature of living with pervasive propaganda is that it is hard to see or consider how your life would be different without the propaganda, and that’s what makes it so dangerous.
> is there any research being done on evaluating a political, and other, bias in a model or is it just being all put in the bucket of hallucination?
It’s a better question, and again one that we should ask regardless of the model’s origins.
Wouldn’t have more to do with propaganda that reinforces and caters to your cognitive biases being less detectable than propaganda that doesn’t? Even inside America, I’m pretty resistant to FoxNews propaganda but if CNN has any, it isn’t registering much on my propaganda detectors.
Yes, certainly. When I say that foreign propaganda is easier to detect, it is because of the biases you mention.
And yes it makes sense that you don’t see the propaganda in the news you watch. My dad is a very smart man who has always liked Fox News and he genuinely can’t see the propaganda in it.
There are also ways that both Fox and CNN share the same views, and this narrow window of thought on the specific subjects they share in common is a key aspect of Chomsky’s Propaganda Model. In areas that get the left and the right emotionally charged at each other, there can be a pretty wide range of opinions expressed. And then in areas that affect certain ways that state and corporate power influence our lives, there is often complete agreement and zero discussion of dissenting ideas. These ideas are represented as base assumptions about the fabric of society that are considered so obviously true as to melt in to the back drop of the discussion, and be invisible to regular viewers.
Those are the ideas that are so vital for us to understand. Nothing will ever change as long as we keep arguing about trans people in bathrooms, gay marriage, and Taylor Swift’s political opinions. That lack of change is what the institutions in power want. What we need to understand to truly change things is ideas of radical democracy, worker power, equitable distribution of resources and universal rights for people and nature.
Those subjects will never be seriously discussed on Fox News or CNN.
FoxNews has gone straight tabloid, but CNN still looks somewhat like news. It’s when they get into the tabloidy stories that I begin to see something is off. But CNN is just trying to make money, and ironically, is copying FoxNews’s formula in preaching to the child and stoking biases to rake in that ad money (because at the end of the day, both really only care about making money, and changing minds rather than reinforcing them isn’t very profitable).
At the very least models will exhibit the bias present in the underlying training text and on top of that there will be a bias imposed by those wanting to correct the unwanted bias present in the underlying training text, possibly swinging the pendulum too far in the other side.
I have the feeling you're asking something more specific, something more of a direct interference coming from politics and not just the natural "point of view" about various topics that are present in the chinese training corpora that is understandably different from western corpora.
Do you have anything specific in mind about something that you expect the Chinese government to feed as propaganda that is not already widely being sculpted into the chinese text corpora available on the internet?
> I have the feeling you're asking something more specific...
> Do you have anything specific in mind about something that you expect the Chinese government to feed as propaganda that is not already widely being sculpted into the chinese text corpora available on the internet?
I don't have anything specific, and it doesn't have to be different from "chinese text corpora available on the internet", it's just that these models can become yet another channel of distribution for the "chinese text corpora available on the internet" especially if they are unknowingly/naively picked up and used as the foundation by others to build their offerings.
PRC will definitely weaponize this for mass foreign propaganda, which up until now PRC has been thoroughly deficient in, despite all the reees of 50cents on western net. The reality pre-LLM is PRC propaganda on western social media platforms has been very limited in scale for the simple reason that they are not wasting valuable English fluency to shit post on western platforms enmass. Low 100s-1000s of accounts, most of which target diasphora in spambot efforts, frequently in Chinese. Now that LLM has made it cheap to spam passable English/foreign languages, I'd expect increased volumes of PRC propaganda on western social media where anonymous posting is asymmetrically easier. But then again, they don't need a PRC LLM for that, plenthy of US bad posting on western platforms from international audiences and US herself.
> they are not wasting valuable English fluency to shit post on western platforms enmass
How's the economy of that different for the Russian campaigns? Do they have a larger pool of English fluency to draw from or is the urgency of the operation higher in their case?
A PRC person with English fluency good enough to blend in with native English on western platform has much better job opporunities. Even in PRC, 50c posts are largely civil servants told to write a few perfunctory platitutdes on domestic platforms. The MO is to overwhelm with spam not engage where effort:return is low. Like even Ministry of Foreign Affairs and most of thinktanks that also publish in English can rarely find people to write "casual" English. You'd have to write 1000s of "50c" comments for 1 hour of English tutoring gig. The economics of it doesn't make sense pre LLM.
Does it mean that in Russia of 10 years ago a person with the same english language skills would not be able to find a better job or does it mean that the troll farms pay more? Or is it just patriotism?
(I genuinely would like to learn more about this topic)
I don't know much about RU. I don't know about RU specifically, cusory search suggest RU has 7/144 (5%) english speakers, prc has 10/1400 (0.07%), so thats the supply demand happening behind the scenes. It's why there was so many expats with no mandarin skills randomly making living teaching English in PRC. If PRC citizen has fluent English skills and also speak mandarin, you can charge stupid amounts for private tutoring or work in private sector. I would say most of pro PRC content are either tankies or Chinese diasphora abroad, including non Chinese citizens who don't agree with MSM narrative. There's about 10M in Anglo/FVEY countries, that's large enough base to have people who are organically pro PRC, or rather not anti PRC.
In terms of state directed actions, RU/USSR has longer history/experience with foreign subversive influenc operations. They're not taking out full page ads to post editorials on western news paper like PRC, which is just clunky. Bulk of PRC work is focused on now largely defunct United Front presence in west, or on PRC social media platforms in Chinese to target diasphora in Chinese etc. It's running joke in PRC that PRC foreign propaganda department is staffed by incompetent old guards who can't even out influence anti PRC EpocheTimes.
As others have reported, English and Chinese queries return different replies on topics that are not kosher in China.
What’s the risk that such models could be used for nefarious purposes by providing propaganda/biased/incorrect/… responses that on a cursory glance seem factual.