That’s the one I’m using. So far it’s quite good, and when I gave it and Claude the same programming problem not only did Llama give a better result, when I showed that result to Claude it also said the Llama approach was better.
Claude is already better than GPT on average at coding, so yeah, bad news for OpenAI as Llama is now potentially better at coding.
Of course Meta has a propriety training set of extremely high quality code, so if they are using that, I’d expect them to have vastly superior performance as FAANG production code is better training data than dogshit stack overflow questions to CS homework problems.
I really think whatever boost OpenAI get from their shadow CoT loop is nominal at best, but with 2x+ the amount of compute forcing them to increase prices an absurd amount.
It’s business 101, they just won’t make the revenue to cover those extra tokens and they are now competing against free. The economics do not suggest OpenAI has a path to survival without major breakthroughs in performance AND efficiency.
That's great to hear. I just want to make sure that you're aware that you're not getting the 100% FP16 experience. I guess at 8bit it's still pretty much the same.
