I often have the situation where a design discussion could easily be settled by A/B testing, only there aren't enough events to get statistically valid results in a reasonable amount of time.

My initial reaction is that this is a sign that we should either focus on getting more traffic or that if we can't measure a difference we shouldn't care either way. However this doesn't really wash in low volume / high value businesses, where a single extra sale means $10-20K.

So how do you handle it? I'd like to avoid reverting to long discussions where everyone is speculating about user behavior with no way of validating the outcome.