Its useful to think about natural selection like the R value for a virus. A gene + environment has an R value (like a virus + its environment. e.g. masks, geography, etc), if this value is higher than 1 it will, over time spread into the gene pool, and under 1 it will die out of the gene pool.

If you think a gene proliferated into the gene pool just because, your still saying its R is 1+, only that the environment played a larger part than the gene.. which is probably always the case anyway.