This might be somewhat off-topic but do we really always have to conduct a study first before we can acknowledge and solve a problem that is obvious if you have any common sense?
I agree with that notion completely, but I think one can overdo it and get lost in pedantry and bureaucracy. Imagine you would program your software and do a benchmark test on every single line. That is what it feels like.
I've always liked the warning to not let perfect be the enemy of good. But I also think it's easy to rationalize too much guessing and too little evidence gathering. I find it a really hard balance to strike.
So introspection is replaced by studies that mostly focus on a single aspect of a complex system, analyzed by people who don't know MANOVA and just do p-value fishing in Excel.
