> I fine-tuned GPT-2 on a 50 / 50 mix of OpenWebText and chess games, and it learned to play chess and continued to output plausible looking text. Maybe there’s something interesting to look at there?
To me that suggests investigating whether there are aspects of human culture that can improve chess playing performance - i.e. whether just training on games produces less good results than training on games and literature.
This seems plausible to me, even beyond literature that is explictly about the game - learning go proverbs, which are often phrased as life advice is a part of learning go, and games are embedded all through our culture, with some stories really illustrating that you have to 'know when to hold em, know when to fold em, know when to walk away, know when to run'.
To me that suggests investigating whether there are aspects of human culture that can improve chess playing performance - i.e. whether just training on games produces less good results than training on games and literature.
This seems plausible to me, even beyond literature that is explictly about the game - learning go proverbs, which are often phrased as life advice is a part of learning go, and games are embedded all through our culture, with some stories really illustrating that you have to 'know when to hold em, know when to fold em, know when to walk away, know when to run'.