Methods for the statistical evaluation of a Bt maize field trial (Cry3Bb1)

Thema

This project aims to guarantee standard statistical evaluation and comparability of trials within the maize project group. This includes being involved in the planning of experiments from a statistical point of view (sufficient repetitions, several trial years and locations, sampling, etc.).

As well as being involved at the planning stage, this project will provide the scientists of the group with biostatistical evaluation methods and software.

Summary

When processing the sample datasets from the ongoing project, it was found that only very simple model calculations are successful for the volume of samples available and for average numbers of individual species. When planning subsequent projects, it should therefore be borne in mind that the chances of a successful statistical analysis are low when studying population dynamics over time with low numbers (frequent zero values) and small samples (8.10 per treatment group). A software package was developed and made freely available to users.
 

Experiment description

Various statistical methods are being developed for and made available to the group which can be used for quantitative safety verifications for the various concerns connected with the cultivation of genetically modified crops in field trials, e.g. for non‑target organisms , resistance factors and for specific plant substances. Calculations include the necessary number of repetitions and acceptable variation ranges. Plot-specific soil parameters (covariables) will also be included in the evaluation.

The project team has already assisted at the experiment design stage to ensure that the experiments are statistically reliable and comparable. The experiments must also run for several years, and some of them need to be conducted at several sites.

Suitable software is developed for all working groups in the project group depending on the research topic.

Proof of safety
"Proof of safety" is based on a statistical approach originally developed for clinical trials, whereby the data collected is used to estimate confidence intervals. Proof of safety can be conducted if these values lie entirely within a range which has been defined as a range of "no relevant change" from an ecological perspective. However, a small number of repetitions can result in the various non-target organisms exhibiting different distributions and extremely different variances. Moreover, individual species can be regarded as ecologically more or less significant.

The quality of the proof of safety was estimated for different numbers of plots per variety, different average abundances rates and different fluctuations in abundances rates in a simulation study. Typical safety limits were selected.

Means of including the spatial layout of the plots were investigated using different models with random plot effects or random column and row effects.

Software

The software package provides a standard interface for the various different calculation methods. Programs were developed for the key statistical methods and made freely available to all users under a General Public Licence (GPL).

Despite the provision of the software programs, which make the system more user-friendly, the methods described still need to be carried out by the statistician.

Results

The project group partners were advised when planning their experiments to ensure that the experiments can be evaluated and compared statistically.

Evaluation of data

For the trial design within the Bt maize research group, plot-specific soil parameters (covariables) and results on the frequency of various groups were available from March 2006.

Individual covariables were found to have significant effects on the frequency of most of the species that were present in large numbers. However, the covariables having significant effects varied (sometimes considerably) in terms of number and type between measurements taken at different times and using different methods within the same year. The simplest explanation for this phenomenon is the low number of observations.

For this reason, another method was tested as an alternative. A sample evaluation and a discussion of the results are available.

During the data evaluation, the question of statistical comparisons of biodiversity indicators arose. Biodiversity indicators group species according to their relative frequency. This means that they also record occurrences of the many relatively rare species for which individual analyses using available methods are statistically unsatisfactory. Statistical methods for comparing biodiversity indicators were developed.

Proof of safety
If safety is defined as stringently as it is for drug approvals in the pharmaceutical industry, proof of safety is scarcely possible with the number of plots used. Adequate proof of safety would then be possible only for very common species (average abundance > 20). If broader safety limits are applied, then proof of safety can also be satisfactorily performed for less common species (average abundance > 5) with low variability. However, for rare species (average abundance < 2) proof of safety can only be performed with very low probability using the existing trial design. This study leads to the conclusion that future trials should use a greater number of plots per variety, or alternatively, that quantitative proof of safety should only be sought for species/species groups occurring in large numbers.

A comparison of the models using data from the project group showed that the inclusion of random row and column effects alone produces a significant reduction in standard errors. This can therefore be used as a simple method of allowing for spatial dependencies.

Software
The results have been evaluated and interpreted using the specially developed software.