Isn't your tech fragile? Feels like it'd break/provide false positives easily.
Integration is also not easy, the matrix of possible tech stacks is big.
It doesn't test new edge cases, so it's useful only for regression, which is important but if your testing hygiene is good, which means devs are writing tests, you wouldn't have the need for your product, am i wrong?
Good luck either way. Seems like VCs are gambling at products that save costs.
Good observation, it's challenging to solve these problems, here's how we are going about it -
To reduce false positives - we run the same request twice to eliminate flaky fields in response like current timestamp, mock the downstream dependencies as they behaved in prod env and are providing options to ignore / modify the sampled requests
To make integration easier - we are building on top of opentelemetry which has seen remarkable increase in support across languages / frameworks, which makes it easier for us to support different tech stacks.
Regression testing - our primary goal is to provide regression tests. We have come across two type of teams where this makes sense - companies with low test coverage and companies which high number of micro-services as they find it hard to cover every production scenario in tests
