Wonder if someone would be willing to start an open source project where we coul...

scottmf · on Feb 20, 2023

That’s what Stability AI has been doing… There are already open source LLMs the size of GPT-3 such as OPT and Bloom

ipsum2 · on Feb 20, 2023

No they haven't. Stability AI is funded by the founder and VC money, not crowdsourcing.

polishdude20 · on Feb 20, 2023

There's gotta be something like this already. Like a SETI @ Home type of thing .

mryab · on Feb 20, 2023

There is! See https://petals.ml/ for inference of models like BLOOM-176B over the internet or https://arxiv.org/abs/2301.11913 and https://arxiv.org/abs/2206.01288 that show you how to do pretraining from scratch in the same setting. Disclaimer: I'm a coauthor of these systems (including the one in OP)

exebook · on Feb 20, 2023

Amazing work! If I had a GPU, I'd join. I know similar project for text-to-image: https://aqualxx.github.io/stable-ui/

efreak · on Feb 22, 2023

https://github.com/aqualxx/stable-horde-notebook

My only problem with stable horde is that their anti-cp measure involves checking the prompt for words like small, meaning I can't use a nsfw-capable model with certain prompts (holding a very small bag, etc). That, and seeing great things in the image rating and being unable to reproduce because it doesn't provide the prompt.