When scientists vie for limited federal funding to support their research, they know their proposals will be evaluated based on scientific merits, but other factors come into play, too: a few years ago, the National Institutes of Health asked reviewers to evaluate the overall impact and significance of the work being proposed.
Those seem like incredibly salient qualities to evaluate before handing out funding. No one likes the idea of taxpayer money supporting trivial research.
But Harvard Medical School systems biologist Marc Kirschner has written a thoughtful, critical editorial arguing that those two criteria are “misleading and dangerous,” and “invite exaggerated claims of the importance of predictable outcomes — which are unlikely to be the most important ones.”
Science is, by its nature, hard to predict, Kirschner points out. People run experiments because they do not know which parts of their hypotheses may hold up and which may be incorrect. Moreover, a research result that seems obviously significant and transformative in retrospect, may have seemed like an narrow, niche area before the big breakthrough occurred.
A good example is the discovery of short snippets of RNA that can be used to turn off genes, which were found in part through experiments on microscopic worms; the work won the Nobel Prize in 2006. In short order, it was adopted as a powerful tool by researchers who are probing questions about human health in laboratories all around the world. Selecting research that seems likely to be significant ahead of time might mean passing on profound projects in favor of ones that are more predictable but less important, Kirschner argues.
But given the need to show a return on research investment and the necessity of choosing some projects over others, how should funding be awarded?
Kirschner answered questions by phone about his editorial, called “A Perverted View of ‘Impact,’ ” which was published Thursday in the journal Science.
Q: You note that significance is often taken to mean medical potential. Why is that misguided?
A: People at the NIH have told me . . . they never meant significance means it has to be done with human cells or human beings or mammals or anything like that. They meant it much more broadly. But for reasons that bother them as much as anyone else, it’s being interpreted over and over again to mean direct relevance to human disease. I don’t think there’s anything wrong with saying there’s relevance to human disease. But it’s [become] very, very narrow. So for example, people will say, “You’ve done this experiment on a mouse or frog or drosophila [fruit fly] and that doesn’t mean anything.” . . . People are being forced out of fields for no good reason.
Q: There’s a big push for “translational” medicine, and many research institutions, including the one where you work, have benefited. What’s wrong with this idea?
A: Wrong might be a strong word — maybe not well-supported. First of all, there’s this assumption that we know a lot, and we all want to cure disease, of course, and the missing link is our ability to translate it. And that kind of assumption makes a lot of people feel very comfortable. . . . So you ask the question, is it really true?
No one’s really made the case or proven it by any means that translation is the . . . rate-limiting feature. So that’s where I feel the argument is sort of very flimsy.
Q: What can be done to change the situation?
A: I guess in some sense, my editorial is really addressed to scientists, and I think the scientific community has to really stand up and challenge some of the assumptions being made instead of being so passive.Carolyn Y. Johnson can be reached at firstname.lastname@example.org. Follow her on Twitter @carolynyjohnson.