Hacker News new | past | comments | ask | show | jobs | submit login

I was speaking very roughly. What you do is basically you can peek early just a couple times and stop early if you see p<0.001. You probably won’t, which “leaves you” 0.045 or so to “spend” at the end. Then you have to pretend that when you see p=0.045, that you actually saw p=0.05, and when you see p=0.04999 that it actually said 0.053 or something (non significant at the 0.05 level even though 0.04999<0.05!), so it’s more restrictive at the end… but only by a little!

The adjustment at the end is small, so it’s not a big complication and people don’t have to radically change their interpretations like they would with some other alpha spending approaches. And then there’s the big plus that since you only have to change the end interpretation by a little bit, any secondary analyses don’t get crazy complicated. They probably just carry just a little asterisk, rather than coming with the big “this is simple but super untrustworthy” asterisk of secondary analyses after other alpha spending approaches, or having to do something very very expert.

If you want to read more, look up alpha spending functions in early stopping. I think the one I’m describing might be called Peto? Sorry i’m being lazy.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: