At first glance, that sounds like it could work. From what I've read, there seems to be two main ways to regain some quality with quantization: post-training which happens after, and quantization-aware training which quantizes during training but leaves the activations and gradients full precision
It's great to see, and to know in one's mind how much likely performance and cost will be better in the future.
I know it's fun to work on, but I also want to say THANK YOU for developing open source.