Yeah, that's why I got involved (I was infrastructure at the time) - how can we throw more hardware at it as the kubernetes setup they had wasn't cutting it.
One of the "data scientists" point blank said in a meeting "My time is too valuable to be spent optimizing the code, I should be solving problems. We can always just buy more hardware".
Admittedly the last little bit of analysis was pretty cool, but >>99% of that runtime was massaging all of the data into a format that allowed the last step to happen.
One of the "data scientists" point blank said in a meeting "My time is too valuable to be spent optimizing the code, I should be solving problems. We can always just buy more hardware".
Admittedly the last little bit of analysis was pretty cool, but >>99% of that runtime was massaging all of the data into a format that allowed the last step to happen.