Back in 2008, Sarah Palin mocked government funded, and presumably frivolous, "fruit fly research in Paris, France." She was totally wrong about the value of fruit fly research, but believe it or not, the general point—that a lot of scientific research doesn’t really bear on the world’s most pressing problems—is actually a fair one.
To address that issue, a group of researchers, led by William Sutherland of the University of Cambridge, recently developed a system for determining which research papers are actually helping to solve real-world problems. Sutherland and his team used wild bee conservation research as their test case. Their aim: to find out which research papers on wild bees actually helped conservation efforts.
First they collected a set of 54 different tactics—the researchers call them "interventions"—for protecting wild bees. These were things like educating the public on bee conservation, increasing natural bee habitats on farmland, and planting parks and gardens with bee-friendly flowers.
They then assembled a group of people who actually work in areas related to bee conservation—policymakers, academics, ecologists, representatives of NGOs, and farmers—and asked them to rank these "interventions" in terms of priority.
Finally, they had a crack team of three bee science experts assess the evidence for the effectiveness of each intervention and judge how much (or how little) individual research papers contributed to the development of the interventions.
With all that data, Sutherland and his team could generate a "total impact score" for a given piece of scientific research based on how much it contributed to a particular bee-saving technique, how effective that technique is, and how popular that technique is among the people who might actually employ it.
In this study, it turned out that increasing the amount of natural habitat for bees in agricultural settings and restoring grassland that bees like were rated as the highest possible interventions, so papers that studied these ideas rated higher. Of the 10 highest study topics, raising public awareness was deemed the most useless by bee conservationists. Studying that was about as useful to them as fruit flies were to Palin:
While this doesn’t mean that scientists can drop their work into an algorithm and find out if it’s worth publishing, it does mean that we can now look back at scientific research and see what’s working and driving innovation, and what’s just keeping scientists occupied in the lab for no reason. And it can deflect criticism when politicians say that money devoted to science is wasted.
Because their impact rating system relies on an existing set of solutions to a problem, it’s most applicable in widely studied areas such as biodiversity conservation, climate change, and medicine. But that’s probably fine. Those are the areas where we have big problems right now.