> Think about the epistemological ramifications of this.

This this this this this a million times this. If your experiment with lab rats needs to be done this finely to be replicated, is it really telling you anything about the real world? Because of course, we don't really care about the behavior of laboratory rats, right? Not in proportion to how often we do studies with laboratory rats versus other species of animals. We care because of what it can tell us about biology in general. And if the finding can't even generalize to "laboratory rats whose cages have been cleaned recently," does the study really say what it seems to?

The point, I think, is that you want to control for things that really alter the outcome. It's not that a clean cage means that the end-product no longer works, but you want to compare "didn't have drug" vs "did have drug" not "didn't have drug" vs "did have drug but the fire alarm went off this morning". Unless your effect size is much larger than all the noise, you could easily miss a true result.

Worse, you could get a false result the other way. What if your control group had all been spooked before they were measured a few times? That could make the control group seem worse than they are!

So, let's continue with your example of a drug being given to the rodents. Now, you want to control diet -- you don't want your control group eating differently than your treatment group. This ends up being relatively easy to do with laboratory rodents, they aren't popping down to McDonalds for lunch or lying in their food diaries, like human subjects might. You buy your rodent food from one rodent pellet supplier, and you don't change brands or SKU during the experiment, and voila, control.

And this works fine if the drug you're studying works identically given all rodent diets. But you don't know that it does! You're not controlling for the variable of diet in the sense that you know the effect of the variable of interest across all possible diets. You know how the drug works conditional on one specific rodent diet. And if you're not putting what brand of rodent food you use, and that causes replication failures, that suggests that you don't understand the effect of the drug as well as you thought you did. And if you have an entire field of research that undergoes such replication failures often, it's fair to start wondering how much of what gets called "science" isn't the study of the natural world but the study of the very specific sets of conditions that happen in labs in American and European universities.

Yes but the things they mentioned would be bizarre to control for the other way.

Would you expect the tests to be done, then again but with both rats hearing the person upstairs installing a new microscope, then with a regular fire alarm, etc?

Describing the food given is quite a step away from describing how frequently the fire alarm goes off.

Exactly. The grandparent comment begs even more terrible questions about this whole enterprise of our species we call science than even the original article.