I think they also published the training methodology as well - that others have reproduced, no? The only thing that I'm not sure is their low level nvidia CTX training code was released under the license - but in order for a third party to corroborate the training and testing they would need to have that code (and likely the training data as well) would they not?
How could they publish the terabytes of training data? A million RAR files?
Honestly would that part even be useful? Like I want to know how they did the training so I can repro it with my own set of training data, right?
I mean, isn't that the future? Somebody figures out how to do P2P distributed training and groups can crawl the web training their own open source models?
