Repeatability in Mathematical Biology

The next View from the Pennines comes out in Mathematics Today at the beginning of next month. It illustrates the dangers of changing your mind when writing to a deadline! The theme is repeatability in mathematical biology, and in particular the way in which the biological journals prevent authors from fully specifying mathematical models (only new terms can be published, `old' systems and parameter values are simply referenced. This creates a chinese whispers trail when trying to recreate the models used, and in the two cases I have tried to follow up, the actual system was NOT the system as specified through the references and I needed to contact the authors to get a full description.

So far so interesting (at least to me!), my problem came when I decided, 48 hours before the final deadline for Maths Today, that I did not want to name the culprits, as the problem is instiutional inside the biological community rather than particular to those two groups, and of course I only pursued the problem because their work was so fascinating (and I want to be able to keep corresponding with them!). However, that did rather blow a hole in the article, turning it into an opinion piece rather than a fully argued article. Such is the problem of journalism.

Anyway, here's an excerpt...

[I]t is perfectly reasonable to redefine what is meant by repeatability in biology. The clockwork of physics is not expected and so one might simply demand some sort of repeatability-on-average (cf. chaos in the deterministic world). Alternatively, and this seems to be more and more the case in systems biology, one can move away from the living world (experiments in vivo) and work in culture (in vitrio) or even through numerical simulations (in silica) where cleaner results are more likely to be obtained. But however well biological mechanisms and chemical pathways are understood in isolation, their behaviour, and even their function, may be different in the more holistic and unpredictable environment of real life (as is, of course, understood by the biologists).

Given this obvious tension in biology it would not be surprising if the biological community was more rigorous about repeatability than most science. I am not in a position to comment on the experimental aspects of biology, except to note that in any discipline which is reliant on significant competitive funding there must be pressure to maintain any advantage by being economical with the details of new experimental techniques. However, there does seem to be an institutionalized undervaluation of repeatability in mathematical modelling. I call this institutional because it appears to be driven by the insistence of journals that models and equations should not be written down if they are already published elsewhere. This leads at best to the necessity for`supplementary material' available from the publisher (with all the access limitations for researchers in less well supported institutions or countries) or, a painful game of Chinese whispers through back-numbers of journals (again, creating all sorts of access problems). Because these models are relegated to supporting materials it also means that the value assigned to accurate reporting is diminished and creates a real issue about how biologists using mathematical models can be educated to supply sufficient detail of their models. There is a related issue around `black box' computational modelling, though I have not looked at this in any detail. The obvious danger is that everyone will agree about results using a given `black box', but that the results will be misleading or downright wrong!