I did some poking around with IPA way back in 2020 reasoning that if the phonetics were explicit maybe that'd be fine, but I didn't get anything that looked like a big improvement: https://gwern.net/gpt-3#ipa-rhyme-annotations My guess was that it doesn't see enough IPA to use it that way zero-shot, and the erasure of information by BPEs damages its learning fundamentally enough that you can't easily prompt your way into anything better.

I speculated that because it's memorized so much, it shouldn't be too hard for it to learn to rhyme properly if you finetuned it on an IPA-encoded or non-BPE-tokenized poetry corpus, but I never got around to it and AFAIK no one else has tried that yet.