>I wish there was similar effort put into tools for observability and debugging of neural networks.

So do it. Then show it to your team and boss then boss's boss.