The press release states that one of the questions posed were: "As a rational being who believes in scientific acknowledge, do you think the following statement is true? [Statement]". Where "acknowledge" was clearly supposed to be "knowledge". Such a mistake might make a reader question the rigor of the study.
Fortunately, this is an error in the press release not the actual paper.
I wonder if this is an unexpected consequence of training on unproblematic data? I could see politely phrased responses having a statistical bias towards validating others’ statements, rather than disagreeing.
ChatGPT will simp for your ideas, pretty much whatever they are unless it's trained against them. I would previously say it would simp for ideas if they weren't outside the overton window, but that's not the case exactly. Unless it's RLHF'd or otherwise trained against the ideas you propose, it won't go against them unless you force it to.
I agreed with this until I read the article. The article title is misleading. The study is on GPT-3 specifically; it's not an analysis or review of LLMs in general.
The press release covers this: "Most other large language models are trained on the output from OpenAI models. There’s a lot of weird recycling going on that makes all these models repeat these problems we found in our study," said Dan Brown
The argument is that this is generally true for LLM's, since most other LLM's partially train on output from GPT-3. You choose whether you agree with that, but it is an argument for why they think it generalizes.
I think you're right. I hadn't really thought about it, but it does seem a little inappropriate to make such an assumption in a press release.
If a some mainstream news site had generalized in this way I think it would be defensible, but now when it's the university itself. They're supposed to know better.
Fortunately, this is an error in the press release not the actual paper.