<strike>This article confuses more than it informs.</strike>
The answer to the question whether one can look at experimental data in A/B tests (or by analogy clinical trials) can be answered with yes even though the article doesn't make this very clear. Instead, it gets lost in superficial Frequentist/Bayesian (keyword) name dropping.
The concept of adjusting for early checking is interesting but this article is less useful than just looking at the the original alpha spending paper:
https://pubmed.ncbi.nlm.nih.gov/7973215/
My takeaway is to avoid mixing the frequentist and Bayesian approaches. Choose one method: either follow the frequentist approach and avoid early data analysis, or use the Bayesian approach to compute posterior probabilities once data are available. Mixing the two without expertise can lead to errors.
I don't see what's wrong with Frequentist approach with alpha spending. The downside is one needs to understand alpha spending. But doing Bayesian without understanding it can be just as bad.
The answer to the question whether one can look at experimental data in A/B tests (or by analogy clinical trials) can be answered with yes even though the article doesn't make this very clear. Instead, it gets lost in superficial Frequentist/Bayesian (keyword) name dropping.
The concept of adjusting for early checking is interesting but this article is less useful than just looking at the the original alpha spending paper: https://pubmed.ncbi.nlm.nih.gov/7973215/
For a Bayesian approach: https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.47800404...