Hacker News new | past | comments | ask | show | jobs | submit login

Their A/B testing results are most likely due to chance.

Using the technique from https://www.evanmiller.org/sequential-ab-testing.html

N = 14626, sqrt(N) = 121

T = 45

C = 34

T - C = 11, far below the 121 required to declare the change a success with 95% probability.

T + C = 79, also below the 121 required to declare no winner.




The deadline for YC's W25 batch is 8pm PT tonight. Go for it!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: