Regardless of how low you get your communication overhead, it still exists. It's rare to find people who can both run and test all the infrastructure and model code, and notice that the transformation you apply on line 34876 of file foobar_now_with_added_ml.py is statistically inappropriate for your problem.
That's not even to mention the really hard part, selecting a good outcome variable and appropriate ways to measure the performance of your system once it hits prod.
You can definitely split this stuff between people, but it gets super-linearly harder as you add more people, so it's really incredible to find people who can do both (and honestly, there aren't that many of them (I'd like to say us, but I'm probably not there yet)).
> It's rare to find people who can both run and test all the infrastructure and model code
It's also unnecessary to do so as long as your institutional processes are capable of synthesizing multiple peoples' competencies across multiple disciplines.
How do you think any machine more complicated than a train car was designed? Do my mechanical engineers need to understand the intricacies of avionics?
> it's really incredible to find people who can do both
Absolutely, and I think you'd have a hard time finding someone who disagrees. But you made a very strong assertion about a "need" which requires much stronger arguments to support.
> It's also unnecessary to do so as long as your institutional processes are capable of synthesizing multiple peoples' competencies across multiple disciplines.
You clearly work at much better run companies than I do ;)
I sincerely wish you luck! Yeah terrible managers are awful. The feedback loop of "my boss won't do shit to help me so I'll figure it out on myself" to "I'm competent in every single technology here and could run the show myself with enough time" to "management recognizes my skills and puts everything on my plate even though I have no time" is pretty dangerous.
That's not even to mention the really hard part, selecting a good outcome variable and appropriate ways to measure the performance of your system once it hits prod.
You can definitely split this stuff between people, but it gets super-linearly harder as you add more people, so it's really incredible to find people who can do both (and honestly, there aren't that many of them (I'd like to say us, but I'm probably not there yet)).