This was based on the training of GPT-3. They mention GPT-4 only in the context of the AI they used to facilitate writing the paper itself.

I'm not sure the scale of 2024 models and usage was influential in that paper at all.