Hacker News new | past | comments | ask | show | jobs | submit login

Um, on your point about the paper being an opinion piece, and thus not important, I would respectfully suggest that you are wrong. The author is John Ionnadis and he has published a number of meta-analyses demonstrating this problem. http://scholar.google.co.uk/scholar?hl=en&q=JPA+Ioannidi... Check out all the articles on the first page, he's been working in this area for a long time.

The author of the piece also identifies a massive, massive problem with science. The problem is essentially, tenure is hard to get, and many people want it. Tenure is dependent on highly cited papers in good journals. These journals have a bias toward significant results which pushes scientists to data-dredge looking for any significant result.

His system of pre-registration (which could be done on the ArXiV model, no government required) would allow us to have a much better idea of the things we're trying to study.

My field (psychology, though I suspect the problem is just as bad in other areas) has a massive problem in this area, as a counter-intuitive result gets published in a good journal, while high quality replications which show no significant effects either don't get published or get published in a much lower ranking journal.

This kind of registration becomes even more important when there are commercial interests riding on the outcome of a study, as in clinical trials. This (and the scandals) is presumably why such a system exists for clinical trials.

Finally, I don't see how registering studies and their designs before running the tests creates a centralised approach to science. The research is all planned beforehand (for ethics committees at the very least), so there's no extra work involved. It also increases our trust in results, as the ones which make it through were predicted in advance, while the weird findings can be replicated.

On a personal level, I can see the pressures to hypothesise after the results are known (HARK) and with tenure decisions looming, I can understand why people do it. Its horribly wrong, so this system would at least cut down on that behaviour.

Full disclosure: almost every time I carry out research, the results are opposite to what I expect, so I am a somewhat biased participant in this debate.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: