> whether an effect is statistically significant ("does the effect exist?") and whether it's significant ("does the effect matter?")
The more accurate terms to describe this is whether an effect is significant (i.e., "do we have enough information to be able to claim it is different from zero?") vs. whether an effect is relevant (i.e., "is [our estimate of] the size of the effect meaningfully different from zero?").
Statistics can only address the first issue (effect significance); the second issue (effect relevance) requires domain knowledge beyond statistics (in some contexts, a difference of 0.01 units can be irrelevant, while it may be relevant in other contexts).
The more accurate terms to describe this is whether an effect is significant (i.e., "do we have enough information to be able to claim it is different from zero?") vs. whether an effect is relevant (i.e., "is [our estimate of] the size of the effect meaningfully different from zero?").
Statistics can only address the first issue (effect significance); the second issue (effect relevance) requires domain knowledge beyond statistics (in some contexts, a difference of 0.01 units can be irrelevant, while it may be relevant in other contexts).