A lot of these points apply to non-RL and non-Deep-Learning projects as well. I'm currently working on a machine learning project that has nothing to do with RL, neural networks, Tensorflow, or GPUs, but I have encountered many of the same surprises and have converged on many of the same solutions and insights. Here are the ones that I think transfer over to general machine learning projects:
- Implementation time vs debugging time: I've spent vastly more time debugging and rewriting. Furthermore, I was surprised at how much time I spent even setting up infrastructure to allow debugging. Corollary: Don't take shortcuts in initial implementation. Literally every hack I put in in order to get something working quickly has come back to bite me and I've had to rewrite and refactor every one of them.
- Thinking vs doing, and being explicit: There are too many degrees of freedom to just try random shit. One of the biggest tools I found (like the author) was to write everything down. I have four+ notebooks filled with literal prose conversations with myself interspersed with diagrams and drawings. I credit that with getting me past many mental roadblocks.
- Measure and test: Without unit and integration tests, I'd probably go in circles forever. It's too easy to change something to fix a problem and have that break something else. I now have a test for even the most mundane assumptions in my model and my code, after too many times realizing "oh, before I changed X I could assume Y was true but that no longer holds".
I feel like engineering for machine learning requires a different skill-set than traditional programming and even traditional AI programming. Taking a very calculated and deductive approach like the author suggests seems to be crucial.
When I was teaching machine learning classes, the biggest hurdle I had to get a lot of students over was their history and expectation that they can implement something, fix the compiler errors, fix the runtime crashes, and then just declare victory. Machine Learning is basically the study of getting a computer do give you an answer you don't know, so you have to be in this constant state of defensiveness, never quite trusting that things are correct.
100% agree, but a key point is: any domain or environment that requires you to program this way (and it's not just ML, for sure), will take surprisingly long to implement anything. There were real gains in productivity from being able to shift out of the mainframe/punch-card era (think long, then test) to the laptop era (try things as part of thinking, in a quick iterative loop). When we have to give that up, things will take a lot longer than the experience of recent decades would cause you to expect.
- Implementation time vs debugging time: I've spent vastly more time debugging and rewriting. Furthermore, I was surprised at how much time I spent even setting up infrastructure to allow debugging. Corollary: Don't take shortcuts in initial implementation. Literally every hack I put in in order to get something working quickly has come back to bite me and I've had to rewrite and refactor every one of them.
- Thinking vs doing, and being explicit: There are too many degrees of freedom to just try random shit. One of the biggest tools I found (like the author) was to write everything down. I have four+ notebooks filled with literal prose conversations with myself interspersed with diagrams and drawings. I credit that with getting me past many mental roadblocks.
- Measure and test: Without unit and integration tests, I'd probably go in circles forever. It's too easy to change something to fix a problem and have that break something else. I now have a test for even the most mundane assumptions in my model and my code, after too many times realizing "oh, before I changed X I could assume Y was true but that no longer holds".
I feel like engineering for machine learning requires a different skill-set than traditional programming and even traditional AI programming. Taking a very calculated and deductive approach like the author suggests seems to be crucial.