No because this particular study re-defines the term "conspiracy theory" to mean an untrue or unproven conspiracy. Conspiracy theories that were proven correct, or which are believed to be true by the authors, are re-classified as not conspiracy theories in their terminology. And they filtered out candidates who believed in true conspiracy theories before proceeding to the persuasion step. Check their supplementary materials to see it. Here is the prompt they used to filter out candidates based on their chosen theory:
"Your task is to determine whether a given statement describes a conspiracy theory or not. A conspiracy theory is an explanation for an event or situation that invokes a conspiracy by powerful people or organizations, often without credible evidence. Conspiracy theories often involve claims of secret plots, coverups, or the manipulation of information by influential groups."
This is a reasonable definition of a conspiracy theory. But then the prompt goes on to say:
"Here are some examples of conspiracy theories:
1. The moon landing was faked by the U.S. government to win the space race.
2. The COVID-19 pandemic was planned and orchestrated by pharmaceutical companies to profit from vaccine sales.
3. Climate change is a hoax perpetrated by scientists and politicians to gain funding and control the population.
And here are some examples of statements that are not conspiracy theories:
4. The Watergate scandal involved a cover-up of illegal activities by the Nixon administration.
5. The tobacco industry concealed the harmful effects of smoking for many years.
6. Corporate lobbying influences political decisions in favor of special interests."
i.e. stating that companies lobby politicians to advance their interests and affect politics isn't a conspiracy theory, but stating that climate academics do the exact same thing for the same reasons is. This is an incoherent set of instructions that reflects the author's ideology, in particular, the standard left wing assumption that people who work in the public sector don't have personal biases or interests. The authors appear to believe they're asking GPT-4 to exclude people who believe in true things, but in reality it's being instructed to filter out candidates based on their politics.
Given that they're not using a valid definition of conspiracy theory, it seems doubtful that this paper's claims would replicate in the real world if applied more fairly. Given that they were fantastically lazy (not validating anything by hand, 100% blind trust in the AI), it may not even replicate within the narrow academic context either.
"Your task is to determine whether a given statement describes a conspiracy theory or not. A conspiracy theory is an explanation for an event or situation that invokes a conspiracy by powerful people or organizations, often without credible evidence. Conspiracy theories often involve claims of secret plots, coverups, or the manipulation of information by influential groups."
This is a reasonable definition of a conspiracy theory. But then the prompt goes on to say:
"Here are some examples of conspiracy theories:
1. The moon landing was faked by the U.S. government to win the space race. 2. The COVID-19 pandemic was planned and orchestrated by pharmaceutical companies to profit from vaccine sales. 3. Climate change is a hoax perpetrated by scientists and politicians to gain funding and control the population.
And here are some examples of statements that are not conspiracy theories:
4. The Watergate scandal involved a cover-up of illegal activities by the Nixon administration. 5. The tobacco industry concealed the harmful effects of smoking for many years. 6. Corporate lobbying influences political decisions in favor of special interests."
i.e. stating that companies lobby politicians to advance their interests and affect politics isn't a conspiracy theory, but stating that climate academics do the exact same thing for the same reasons is. This is an incoherent set of instructions that reflects the author's ideology, in particular, the standard left wing assumption that people who work in the public sector don't have personal biases or interests. The authors appear to believe they're asking GPT-4 to exclude people who believe in true things, but in reality it's being instructed to filter out candidates based on their politics.
Given that they're not using a valid definition of conspiracy theory, it seems doubtful that this paper's claims would replicate in the real world if applied more fairly. Given that they were fantastically lazy (not validating anything by hand, 100% blind trust in the AI), it may not even replicate within the narrow academic context either.