The notion of a central registry and government regulation for science is horrific and backwards no matter what the initial intention is. Why not a distributed, peer-to-peer approach for scientific publishing where all articles are anonymously accessed and linked to each other?
Because that, in no way, will prevent researchers from trashing studies until they get the results they want, or fishing for any significant correlation after the fact as if that's what they were looking for all along.
Some sort of validation mark that a study announced itself before being started would at least show me which ones I should ignore - because IMO this type of thing is ruining medicine, esp. pharma.
At this point, there's very little evidence that some major drugs, classes of drugs, and treatments work at all, and that's both scary and a systemic financial drain.
I think it's a rather misleading post. He cites a paper from PLoS Medicine, and I suspect that those without a good background in research would rather quickly jump to the conclusion that "it's in a science journal, so it must be (very likely to be) true." However, it should be noted that this is an essay, which PLoS Medicine defines as opinion pieces on a topic of broad interest to a general medical audience.
More importantly, lets look at this from a pragmatic point of view. Are there things that can be done better in science? Absolutely. Does science somehow 'not work'? Not at all. Does it need fixing? No, why should it? Anyone who suggests that science is fundamentally broken must've lived under a rock for the past decades, or at least hasn't been keeping an eye on what science has achieved.
I also disagree with the blog author's 'solution' to the problem. He (or is it a she? I don't know) argues in favor of a centralized approach to science, whereas one of the reasons science is so successful is because it's so decentralized!
Um, on your point about the paper being an opinion piece, and thus not important, I would respectfully suggest that you are wrong. The author is John Ionnadis and he has published a number of meta-analyses demonstrating this problem.
http://scholar.google.co.uk/scholar?hl=en&q=JPA+Ioannidi...
Check out all the articles on the first page, he's been working in this area for a long time.
The author of the piece also identifies a massive, massive problem with science. The problem is essentially, tenure is hard to get, and many people want it. Tenure is dependent on highly cited papers in good journals. These journals have a bias toward significant results which pushes scientists to data-dredge looking for any significant result.
His system of pre-registration (which could be done on the ArXiV model, no government required) would allow us to have a much better idea of the things we're trying to study.
My field (psychology, though I suspect the problem is just as bad in other areas) has a massive problem in this area, as a counter-intuitive result gets published in a good journal, while high quality replications which show no significant effects either don't get published or get published in a much lower ranking journal.
This kind of registration becomes even more important when there are commercial interests riding on the outcome of a study, as in clinical trials. This (and the scandals) is presumably why such a system exists for clinical trials.
Finally, I don't see how registering studies and their designs before running the tests creates a centralised approach to science. The research is all planned beforehand (for ethics committees at the very least), so there's no extra work involved. It also increases our trust in results, as the ones which make it through were predicted in advance, while the weird findings can be replicated.
On a personal level, I can see the pressures to hypothesise after the results are known (HARK) and with tenure decisions looming, I can understand why people do it. Its horribly wrong, so this system would at least cut down on that behaviour.
Full disclosure: almost every time I carry out research, the results are opposite to what I expect, so I am a somewhat biased participant in this debate.
If science could be done better, yes, we should fix it. The author identifies a very real problem - publication bias and data mining, and proposes a solution, namely pre-registration of experiments.
Only one of his mechanisms for enacting the solution (#1, "clinicaltrials.gov option") is centralized.
> Anyone who suggests that science is fundamentally broken must've lived under a rock for the past decades
There are many different fields in science, each of which needs its own rules / conventions. For example, physics is a field with a really long history, where the scientific method has been perfected over the centuries by a number of very smart people doing thousands of experiments. Still, there are errors that take some time to be resolved (the recent neutrino blunder - although it should also be said that it was resolved precisely because physics is such a well-organized field of science).
Other sciences, e.g. medicine, are much younger (the scientific method being formalized only in the last century), mistakes are more costly (people might die), experiments take longer and the effects of new discoveries are not visible immediately (could take a whole generation). E.g. I still wonder how will the prevalence of mobile phones affect the fertility of the people who have been carrying a mobile phone in their pocket (next to ovaries/testicles) for their whole lives, but there is no answer and there won't be one for at least another 20 years.
Only the future generations will be able to assess whether our science is broken or not, and whether we're doing anything to fix it.
The author doesn't claim science is fundamentally broken, and i believe his suggestion is a valid one. article . I don't know about other fields, but certainly in my field, neuroscience, there is a lot of work that is being repeated, sometimes unnecessarily, and a big hunt for sensational titles that will get to the top journals. I understand that there's competition among labs, but sometimes this leads to similar studies being conducted independently, instead of combining forces for a single more in-depth study. That leads to a situation where there are no broad-scale projects (of the level of the human genome project or LHC) being conducted by academics (but private institutions do conduct them). His proposal about a central registration mechanism is not perhaps a panacea, but it's not a bad idea.
