Oh for finetuning - we do have some code for MoE finetuning for Qwen at https://github.com/unslothai/unsloth, but we haven't yet announced it yet!