Petals doesn't train new models, it only runs BLOOM in a distributed way.

You can finetune with it. If you want a more generic framework you can use hivemind[1] which is what petals uses, but you'll have to create your own community for whatever model you're trying to train.

https://github.com/learning-at-home/hivemind

Obviously fine-tuning is just a special (easier) case of training.

They are working on complete training of large models too: https://github.com/yandex-research/swarm