One aspect of testing they leave unsaid is you identify your users (cookie, most commonly) to make sure each user always gets the same experience. That's why your numbers are all based on unique users, not merely users.

Their experience will still change once their cookie expires, but that amount of time is completely under your control.