hence GPT-2 or its near successors are not dangerous
FWIW, my post that's linked here doesn't claim that GPT-2 won't have dangerous successors, just that GPT-2 itself is not very dangerous, and that this doesn't vary much with the model size. The point is not about the five-to-ten-year trajectory of natural language generation research, which could go all sorts of unexpected places -- indeed, GPT-2 itself was such an "unexpected place" from the perspective of a few years ago. The point is a narrower one about OpenAI's staged release plan, which assumed different levels of risk, or at least different prior distributions over risk, for different sizes of the same model.
I agree that
Architectures that tune text creation to create clickbait titles with underlying goals are going to get worked on and thought about and tested
but that is a distinct discussion, one about future research milestones enabled by the relative success of generation by sampling from transformer LMs. My claim is that significant further milestones will be necessary (not that they won't happen); there are big and relevant limitations on sampling from currently existing transformer LMs, and simply making the transformer LMs bigger does not remove these. In particular, these limitations make it unlikely to be cost-effective to make clickbait or propaganda by curating and cleaning samples from such a model.
If you doubt this, I'd encourage you to actually go through the exercise: imagine you want to generate a lot of text of some sort for a specific real-world purpose that would otherwise require human writers, and try out some strategy of sampling, curation, and (optionally) fine-tuning with one or more of the available GPT-2 sizes. If your experience is like mine, you'll draw the same conclusion I've drawn. If not, the results would be fascinating and I would honestly love to see them written up (and OpenAI might as well).
FWIW, my post that's linked here doesn't claim that GPT-2 won't have dangerous successors, just that GPT-2 itself is not very dangerous, and that this doesn't vary much with the model size. The point is not about the five-to-ten-year trajectory of natural language generation research, which could go all sorts of unexpected places -- indeed, GPT-2 itself was such an "unexpected place" from the perspective of a few years ago. The point is a narrower one about OpenAI's staged release plan, which assumed different levels of risk, or at least different prior distributions over risk, for different sizes of the same model.
I agree that
Architectures that tune text creation to create clickbait titles with underlying goals are going to get worked on and thought about and tested
but that is a distinct discussion, one about future research milestones enabled by the relative success of generation by sampling from transformer LMs. My claim is that significant further milestones will be necessary (not that they won't happen); there are big and relevant limitations on sampling from currently existing transformer LMs, and simply making the transformer LMs bigger does not remove these. In particular, these limitations make it unlikely to be cost-effective to make clickbait or propaganda by curating and cleaning samples from such a model.
If you doubt this, I'd encourage you to actually go through the exercise: imagine you want to generate a lot of text of some sort for a specific real-world purpose that would otherwise require human writers, and try out some strategy of sampling, curation, and (optionally) fine-tuning with one or more of the available GPT-2 sizes. If your experience is like mine, you'll draw the same conclusion I've drawn. If not, the results would be fascinating and I would honestly love to see them written up (and OpenAI might as well).