HONEY BEES, Bt CROPS, AND THE ROLE OF META-ANALYSIS IN RISK ASSESSMENT
Michelle Marvier
July, 2008

As the world's most abundant and widespread pollinator, honey bees (Apis mellifera) play a critical role in our food security and make an important contribution to the human economy. Because of their global importance to agriculture, there has been a great deal of consternation surrounding the widespread recent decline of honey bee populations¹. Amid the sometimes wild speculation about what might be causing these declines, it has been suggested in several popular media outlets that pollen from genetically engineered Bt crops might be poisoning honey bees^{2, 3}. It should be emphasized, however, that there are not any scientific publications indicating that Bt crops have anything to do with honey bee declines. All studies performed to date support the contention that Bt toxins, called Cry proteins, are in fact toxic to only a narrow range of insect groups. For example, the Bt transgenes that have so far been incorporated into crops such as corn and canola are toxic to Lepidoptera (butterflies and moths) or Coleoptera (beetles). Although there is the possibility that these Cry proteins are toxic to certain taxa, such as caddisflies (Trichoptera), that are closely related to the target group⁴, hymenopterans (bees, wasps, and ants) are not closely related to beetles or butterflies and moths, and in the laboratory no toxicity test has ever shown Cry proteins to cause any harm to hymenopterans.

Nonetheless scare stories about Bt crops and bees keep circulating. The question is whether, given the absence of evidence implicating Bt crops in honey bee declines, is there anything more that can be done to eliminate this worry once and for all? In order to fully lay to rest the worry about Bt pollen and honey bee declines, there is indeed an additional analysis that needs to be done—one that in a quantitative way "adds up" all evidence from independent experiments that have assessed impacts of Bt toxins on honey bees. This additional step, called meta-analysis, has gained prominence in clinical trials and the medical arena, where one has to be very careful before deciding that a treatment is relatively risk-free. The crux of meta-analysis is the realization that an absence of significant effects in a collection of individual studies is not necessarily as convincing as it might first seem. The problem is that the individual risk assessment or toxicity studies may be poorly replicated and thus have low statistical power. For example, a study might expose three groups of honey bees to a Cry protein incorporated into a standard diet and three groups of honey bees to a control diet, lacking the Cry protein. No matter how many honey bees are in each "group," the replication of the study is only n = 3. With such low replication, only a large and very consistent difference between the two treatments in the survival, development, or growth of the honey bees could be detected as statistically significant. The weak statistical power of these studies means that a finding of no significant effect is not very convincing.

A sample size of just three replicates per treatment might sound unrealistically low, but the reality is that three replicates meets the EPA standards for assessing risks to honey bees and other nontarget invertebrate species. Many studies do use more than three replicates, but in general the level of replication used in industry studies for nontarget invertebrates such as honey bees is only n = 2 – 6 replicates per treatment⁵. However, wouldn't it be reassuring if there were a dozen or so of these poorly replicated studies, all indicating no significant effect of Bt pollen? The answer is no. In fact, a simple tally of the results (the number that found, versus didn't find, significant effects) from a collection of weak studies is not much more convincing than the findings of each individual study on its own. Among statisticians, such tallies are called "vote counts" and if one thinks about it a bit, it is pretty obvious that even a dozen studies, all with poor replication, finding no effect would not constitute convincing evidence that no effect actually exists.

Meta-analysis to the rescue
Fortunately, meta-analysis provides an alternative to vote counting. By statistically combining the observed differences between treatments and controls across a group of independent studies, and weighting the results of each experiment by the variance in the data, one comes up with an estimate of the general effect size across experiments. This resultant effect size is much richer than simply stating that 9 of 11 experiments or even 11 of 11 experiments found no significant impact on honey bee survival. It is possible that, by applying meta-analysis to a set of poorly replicated studies, a more reassuring picture may emerge. Of course, it is also possible that a meta-analysis will draw out small but potentially biologically important effects that went undetected by any individual study.

Meta-analysis of clinical trials has revolutionized health care, and in ecology, meta-analysis has produced some of the clearest general evaluations of predation, competition, and herbivory. In conservation of endangered species, meta-analysis is just now being adopted as a way to assess the effectiveness of alternative management actions. Risk assessment of genetically modified crops should similarly embrace this paradigm.

To facilitate meta-analysis of risk assessments that have examined the nontarget effects of Bt crops, my colleagues and I have created a searchable database, publically available at http://delphi.nceas.ucsb.edu/btcrops/. Between April 2007 and February 2008, this database was queried by visitors from 215 unique IP addresses, and we hope scientists around the world will use the data to ask questions that summarize all of the evidence available about the nontarget effects of Bt crops. Initial meta-analyses of the field (as opposed to laboratory) studies included in this database have recently been reported⁶, but to date there have been very few field studies that have recorded the abundance of honey bees.

How about those honey bees?
The question of whether Bt crops might be contributing to honey bee declines was clearly in need of a more definitive answer. To provide that answer, and also to demonstrate the general utility of a meta-analysis approach for risk assessment, my coauthors and I assembled and analyzed a collection of 39 independent assessments (appearing in a total of 25 separate publications or unpublished industry reports) that examined the direct effects of Bt Cry proteins on the survival on honey bee larvae and adults in a laboratory setting⁷. Our meta-analysis of the data from these studies revealed no adverse direct effects of Bt Cry proteins on the survival of either larval or adult honey bees. I should note, however, that the studies synthesized in our meta-analysis were all laboratory experiments, so there is still some possibility that different results might be seen in the field, where the stresses of weather, disease, and so forth might alter the susceptibility of honey bees to Bt toxins. On the other hand, the studies from which we drew these data were all so-called Tier I studies that exposed honey bees to extremely high concentrations of Bt toxins—at least an order of magnitude greater than the concentrations that bees would encounter in nature. Given these caveats, we believe that our meta-analysis strongly supports the conclusion that the Cry proteins expressed in the current generation of Bt crops are unlikely to have adverse direct effects on honey bees.

Evidence-based risk assessment
We often hear about all the experience and trials and tests that have been done to assess the safety of genetically modified crops, but there is no single place for concerned citizens or even scientists to turn to in order to see if they are themselves convinced by the accumulated evidence. We believe that creating large open-access databases that include data from all relevant risk assessment studies is the future of risk assessment. With such databases at hand, the power of meta-analysis and of a global community of scientists can be turned loose.

A formal synthesis of what scientists had learned about the effects of Cry proteins for honey bees had been lacking until recently. Once we put together the database, the meta-analysis was straightforward. However, there are many other questions about impacts on other groups of organisms, or concerning how effects might correlate with differences in everything from body size and growth rates to the experimental methods used to assess the risks. The construction of databases detailing the methods and results of all sorts of risk assessment studies, followed by the creative application of meta-analyses to these data, offers the clearest path to the sort of transparent cost-benefit analyses that society deserves.

Meta-analyses have the potential to move the debate about the safety of genetically modified crops beyond a situation in which competing sides argue that "study X shows this" only to be countered with "yes, but studies y and z show the opposite." Indeed, no single study should, by itself, be taken too seriously until other studies have confirmed the findings. Yet there are so many scientists doing so many different experiments and risk assessments that the information has the potential to overwhelm decision makers or cause the debate to zig-zag around. If meta-analyses and large databases of completed studies were to become a routine part of risk assessment, then there would not be the distraction of single experiments capturing media attention and inappropriately alarming or comforting the public and policy makers. An investment in the creation and maintenance of risk assessment databases will have high payoff in terms of improved transparency, increased public confidence in the process, and more rapid advancement of scientific understanding.

The creation of the "Nontarget effects of Bt Crops" database was supported by EPA grant CR-83214701 awarded to Michelle Marvier and Peter Kareiva of Santa Clara University.

Michelle Marvier
Associate Professor of Biology and Environmental Studies
Santa Clara University
mmarvier@scu.edu