Do you have any references/explainers for that approach? Would be interested to ...

sgt101 · on Nov 18, 2020

The best I have is this one I wrote a long time ago:

https://www.aaai.org/Papers/Workshops/1999/WS-99-06/WS99-06-...

But, I apologize ! It's a bit pimped up compared to the one liner above, I think step 7 in section 4.3 is what I was thinking of :) I did laugh when I dug it out, as I have been working on the first bullet in the conclusion this week!

FlyingLawnmower · on Nov 17, 2020

Check out this work from Rich Caruana & collaborators on model compression: http://www.cs.cornell.edu/~caruana/compression.kdd06.pdf

which was a precursor to the model distallation work from Geoff Hinton: https://arxiv.org/abs/1503.02531

underaxon · on Nov 22, 2020

I've made some experiments that you can check out here:

http://www.clungu.com/Distilling_a_Random_Forest_with_a_sing...