What we really need to find a way to support long-term, high risk projects driven by young scientists with breakthrough ideas. At the moment all scientists have to live from grant to grant which makes it near impossible to go after the really risky projects. If you try a difficult project and fail then you are out of a job.
We also, because of the wonderful system of peer review, pretty much only give grants to old established researchers with long track records. These scientists are great at running mini-empires, but not so good at coming up with the really novel ideas that we so need.
Having a lottery is a good idea and one I support for grants in general. The grant review process should be simply "is this idea viable and worthy of funding - yes or no”. If yes put it into the lottery and fund as many good grants as you have resources. Peer review works well for this type of sorting, but it is terrible for trying to pick a top 10% idea from a top 15% idea. It just isn’t possible for even the best scientists to be able to do this consistently.
The problem with fellowships is that the longest I know of only go for 5 years and most of the junior ones are only for 3 years. This is just not long enough to take the chance on failure.
If you want to encourage young talented people to tackle really hard and high risk problems you need to provide funding for at least 10 years.
I would go as far as adding a restriction that recipients of these high risk fellowships can’t publish in anything but the absolutely top tier journals (Nature, Science, Cell, etc) for the first 7 years so they don’t get distracted from the high risk work that other scientists can’t afford to do.
In my opinion we are missing out on great discoveries because we allow a small number of humans determine if a scientific research question is worthy of funding. If a single scientist thinks an idea is worth doing and can find the resources to support it, she should do it.
We will move to a system where we only ask 'can this question be answered by the scientific method?' If we use the scientific method to answer these questions we will uncover secrets of the universe that would never survive in the current system humans have designed.
The best scientists do not design their experiments with a goal of publishing in top tier journals. And as a community, we should not be encouraging this behavior.
The best scientists investigate questions out of pure curiosity and as a side effect publish the most impactful research.
Young talented people are already tackling really hard and high risk problems. If this is truly what young scientists want to do, they will find a way. 10 years of guaranteed salary is not the solution, but giving everyone the opportunity to be a scientist may be.
>Young talented people are already tackling really hard and high risk problems. If this is truly what young scientists want to do, they will find a way. 10 years of guaranteed salary is not the solution, but giving everyone the opportunity to be a scientist may be.
I love your idealism, but having climbed the greasy pole to academic tenure (I have since left science) I can say that you really can’t afford to work on the high risk projects. When you are a junior scientist the more senior scientists control what you can work on and they want you generating publications. When you are a post doc you can’t afford to not get publications out and hence you have to work in areas which will produce results within 2 years. Once you are a senior scientist you are forced to work on projects that will generate x number of papers within the 3 year grant period or else you wont get any further funding. At every stage of your career you are effectively forced to work on projects that are guaranteed to generate publishable results within the next 2 years.
The reason I suggested that the scientists awarded one of these 10 year fellowships are not allowed to publish in anything but the top journals is to remove the pressure of publishing off the fellows and allow them to concentrate on solving the difficult problems. The only reason I suggested an exception for the top tier journals is in rare case the fellow made a lucky breakthrough - we don’t want them sitting on some important result for years just because they not allowed to publish.
I have many high risk/high reward projects that I would have loved to have worked on when I was a professional scientist, but I could not afford to do so. I knew if I failed to produce consistent publishable results I would be out of a job. Even once I had tenure I could not in all honesty ask my students or post docs to work on projects with a significant risk of failure. We have created a system where we have a high probability of making incremental progress, but almost no chance of big breakthroughs.
I've always wanted to run an experiment where NIH or NSF split the proposals randomly 50/50. Group A gets peer reviewed and awarded the way the current system works. Group B gets peer reviewed and the top half of proposals gets put into a lottery and awarded at random. I am willing to bet the outcomes and impact of Group B will always be better than Group A.
Cindy I suspect you might be right - especially if we measure for breakthrough discoveries.
The problem you face running such a trial is those scientists that control the current system are not too interested in trying anything that might take away their control. They love to mis-quote Churchill and claim their is no better way than the current system, but they are none too keen to try any alternatives.
I'm currently working through preparing for a candidacy exam, and I've found a number of errors on a Physical Review article...yes, a Physical Review. They aren't the crux of the paper, but they are fundamental equations that they apparently implemented in their code (of course, I don't have access to that code), and these updates are the crux of the paper. If the equations they wrote are wrong, how can I trust their results?
Recently a senior researcher admitted to me that from a journal he referees for, he was referee-ing for a paper and found 10 issues with it. However, the editor came back to him and told him accept it.
You're damn right it's stacked, and sometimes "long track records" just means cosy with the publisher. It's not like there isn't corruption elsewhere in the world, but young scientists do not have start-ups. They do not have opportunities or the means to go against the grain and get into those risky projects unless it's after the decade or two of getting cosy with others in the academe and building those mini-empires. You're very right they need a venue, an accelerator for grad students, I suppose.
Just curious, have you asked the researchers for their code? I've found that some people are willing to share their code and others aren't (some are just shy and embarrassed about how ugly it is, others have less justifiable reasons like it gives them a competitive advantage) Have you discussed your findings with them? Have you considered writing a comment outlining their errors (you might talk to your adviser about that)?
Refereeing is difficult to do well. On the one hand,in a Physical Review Letter, the article should be more broadly accessible, but on the other hand, it needs to be technically correct. I remember once reviewing a methods paper (different journal) and checking each equation and each integral (and finding a few minor errors) because it wasn't likely to be checked so much once it made it into a black box--but that took a lot of time! For myself, I've never had an editor tell me to "accept" a paper with errors. I have given a report and had the editor make a decision the other way, but that's different then asking someone to amend a report.
As for taking risk, I think the best advice I can offer is to have a portfolio of projects. I'm more senior in my career, but I did the same thing as a graduate student. Some things I work on have little chance of success, others more. This will vary by field and how expensive your needs are--but part of choosing an adviser (especially at the postdoc level) is choosing one that will allow you to explore your own ideas. I feel that my duty to postdocs is to offer some ideas that may work. If they have other ideas that they'd like to explore, then I view my job as to offer my expertise to support them...Micromanaging is too much work!!!!!!!!!!!
>Recently a senior researcher admitted to me that from a journal he referees for, he was referee-ing for a paper and found 10 issues with it. However, the editor came back to him and told him accept it.
I have had this happen. I have rejected papers as fundamentally flawed and the editor has just published the papers unchanged. It make you rather cynical about the whole peer review process.
The SF Bay Area community bio labs (BioCurious and Counter Culture Labs [0]) are considering using this platform to try and fund their iGEM team, researching inducing UV resistance in E. Coli and then optimizing it with directed evolution. Community labs have effectively no funding to operate with, so these types of platforms (Experiment, Kickstarter, etc) are incredibly important for them.
If you'd like to support the SF Bay Area community labs and hobbyist biologists trying to do real research without academic and commercial funding, watch the Experiment.com biology section [1] over the next week or two!
Sadly I'm not in the Bay Area for the next 3-4 weeks and missed the original message :) Also I'm not really the one organizing the BioCurious funding, so maybe not the right person to speak to on that matter...
Nonetheless I'd be delighted to meet you at some point, Experiment.com sounds like a pretty great organization :)
Saying that a 3 min speech inspired 1.2m in investments is almost always silly, even if it was at demo day. It undermines all the work behind that speech, which is sometimes years of work.
Experiment.com falls into an awkward area for my fundraising efforts for longevity science projects. The community I raise from has been funding a few n*$10k scientific projects every year or so for a decade now, and the yearly fundraiser I cheerlead brought in $150k last year, and will hopefully do as well this year (halfway there). We're growing slowly in reach, but would greatly benefit from a Kickstarter-like thing that helps us pull in a larger audience and new faces.
That doesn't seem to be experiment.com yet, or indeed any of the organizations trying to crack the same nut of how you make crowdfunding work for research projects. We wouldn't do any better with them than we do with our own hacked-together infrastructure and communication channels. We need the existence of a large successful science crowdfunding community before it is worth our trying to surf on that science crowdfunding community, and I'm sure we're not the only people with that chicken and egg situation.
Some folk in the community have had luck with indiegogo.com for scientific research projects, such as mouse gene therapy or immune transfer treatments in European labs at a few tens of thousands of dollars, but even there I think it is a case that you are buying infrastructure for the audience you bring to the table, not buying fresh faces and new listeners to persuade to your point of view.
I'm vaguely hopeful that among the competitors in this space someone will come up with something that works and scales and pulls in the crowds, the halo of attention. There are some quite different styles of approach floating around, such as labcures.com that implements a "back a team" strategy, or lifespan.io that'll be launching sometime soon. Perhaps one of them will gain more traction. Experimentation is key.
Eh, I guess I don't mind. Our lever on the chicken-egg problem is that we'll always go out to find the best and most impactful science, and this type of science will always be funded because this type of science is always growing. We'd rather have 1000 $1k projects than 1 $1M project. The halo of small groups is what will bring meaningful scientific results. Caveat to all of this is that there is one project right now that's so far raised $1.55M (http://experiment.com/curebatten) for orphan disease research.
Researchers who use Indiegogo are only using it because 1) they are locked-in by their institutions (a small number of universities coerce their faculty like this) or 2) they can't yet use Experiment because they are outside of the US.
Come to think of it, I don't really think kickstarter or indiegogo provide any sort of broadening of possible audience, as you put it. I mean, I don't think people scan kickstarter's front page looking for things to throw their money at, its generally up to the project managers to publicize, and then kickstarter provides the communication infrastructure once you have that audience pulled in.
What a great idea. Maybe this is what will keep corporate interests out of scientific research, as there's more transparency and diversity in funding sources. I'm willing to bet the reproducibility rate for these studies will be much higher than the ~30% average.
In my opinion reproducibility is a technical problem. Once research is published and peer reviewed in real time, all studies should be reproducible. If they are not reproducible, we should have insight into exactly why they are not and propose a solution.
Sadly the stats aren't that good. From the article "Two years on and with a smart name change, “Experiment” has launched 5,058 projects and funded 336 of them" so still a long way to go - Kickstarter's success rate I think is around 40%. That's still 336 research projects that wouldn't have been funded otherwise! Go Cindy!
The 5,058 projects figure are people who've signed up and started projects. There've actually only been 836 launched projects, with 336 funded and 395 failed. Not sure why that number got reported, perhaps an attempt at "there are 5000+ researchers who've signed up".
Every one has themselves philosophy, they only denotes the project they think worthwhile.
Some projects don't funded only due to the lack of ones with same belief.
But it is really exhilarating that 336 projects are funded
Every research project has an existing community that wants to support it. The challenge is connecting the two. In a truly efficient world all projects would be 100% funded.
Branding. .com is the top-domain. It's synonymous with Internet companies. Without a traditional TLD, your domain doesn't even look like a domain. Instead it look s like a bunch of random.words.smashed.into.one. (Which coincidentally, is a valid domain.)
I like this a lot. Kickstarter for scientists.
It would be very welcome if scientists didn't have to come up with military applications of their research in order to get funding.
To tie a funding round to nothing more than a "spirited 3-minute speech" is silly. If I had to guess, Cindy is a fearless founder who has been working hard for a long time to achieve her vision.
This article's title makes it sound like it's as easy as skipping your Starbucks run to found a company and raise $1M+.
We also, because of the wonderful system of peer review, pretty much only give grants to old established researchers with long track records. These scientists are great at running mini-empires, but not so good at coming up with the really novel ideas that we so need.