Hi - we definitely weren't looking at this full time for two months, just off and on as it failed various test runs, and I had time to look at it.
We have a large suite of tests that integrate with Postgres. Maybe one out of 50,000 database queries would fail. It was hard/impossible to predict which query would fail, since the problem wasn't with any individual query. By your logic, we should throw out our entire test suite. I'm not sure what we're supposed to replace it with.
> By your logic, we should throw out our entire test suite.
That's why I figured I was invoking the wrath of the testing fanatics... that is where my logic leads. :-) I have no idea what your tests look like, maybe they're super useful, but I've worked at places in the past where there were thousands of tests that were frankly pretty useless, and were actually a net negative for a variety of reasons, but to point that out got you labeled as a "cowboy". (To be clear I'm not against automated testing, just automated testing done badly.) If you're running them in a context where you can't debug it, at the very least I would move it in house onto a machine where you can hook up a debugger.
Our tests are pretty useful, they check things like "when you ask the API to submit a pickup, it gets assigned to a driver", and so on. Especially in JavaScript where anything can take any value they've been extremely helpful.
I've been working on applications with javascript for over 10 years.
Not once can I remember a javascript bug where someone passed a value as the wrong argument and it made it into production.
Maybe it has happened, but it's just not a common bug. I just don't remember it.
Like, you can easily write it as a mistake as you're developing but it's obvious as soon as you run it. But if that bug gets in production your dev didn't even bother to run the code to see if it worked as intended.
We have a large suite of tests that integrate with Postgres. Maybe one out of 50,000 database queries would fail. It was hard/impossible to predict which query would fail, since the problem wasn't with any individual query. By your logic, we should throw out our entire test suite. I'm not sure what we're supposed to replace it with.