Is this just fine tuning a cheaper model like Curie on the outputs of a more expensive model like Davinci? I proposed this as a question/idea to an OpenAI employee on Twitter and he talked me out of it.
I think it depends on how the answers from davinci are used. If the answers are used verbatim to fine-tune a cheaper model, then that could be problematic. However if it is used to generate a fine-tuning dataset which is then corrected manually, I don't see the problem.
He didn't give a great explanation. Just kinda kept dismissing it as a bad idea and suggesting other paths.
I don't see why using the Davinci outputs verbatim would be a problem in certain situations. The goal is just to get a fine-tuned cheaper model (like Curie) closer to Davinci performance in some narrow problem domain. Of course it's never going to be as good or broad as Davinci with this approach, but the lower cost may outweigh that. Just surprised more people haven't tried and benchmarked this approach...but I'm no expert here so there is probably a good reason.
