I talked with one or two people were interested in further poetry work but I haven't heard of anything since. The most interesting recent news is someone claiming that InstructGPT can be prompted to do what looks like genuine rhyming, but they didn't share their prompt and I've been too busy to followup.
The vocab tokenizing already mostly works, as you can see from the results. I think the solution really is as simple as switching to a byte encoding like ByT5, and perhaps someone spending 30 minutes to run a few megabytes of text through an IPA phonetic encoder to enrich the training corpus. Rhyming is just not that hard or complex compared to the things neural nets already do. It's purely artifactual. Then one can focus on more interesting and challenging tasks, like curating high-quality corpuses for finetuning.
Yeah, the challenge there is both understanding the word sounds and enough of the world to say something clever.
It shouldn't just help with rhyming, puns that depend on understanding how words sound. One of these models could easily be vastly super human at cross-language puns if they only understood the sound of words. :P
The vocab tokenizing already mostly works, as you can see from the results. I think the solution really is as simple as switching to a byte encoding like ByT5, and perhaps someone spending 30 minutes to run a few megabytes of text through an IPA phonetic encoder to enrich the training corpus. Rhyming is just not that hard or complex compared to the things neural nets already do. It's purely artifactual. Then one can focus on more interesting and challenging tasks, like curating high-quality corpuses for finetuning.