Well, you'd need to train it from scratch to operate on character level. And it would have smaller context, thus lower quality. So if you want same quality, you need much bigger context.

Still, would be an interesting experiment. Gwern swears it would improve stuff, so worth trying and comparing, I guess