Being a Newfoundland expat currently living in Boston during this awful global pandemic, I have kept abreast of the news on both sides of the border. One of the reasons is the desire to travel, so that I can see family, friends and my daughter gets to visit her nanny and poppy. Typical away from home stuff.
As such, the headline “Stanford, Oxford, and Memorial University study cautions against lifting Newfoundland’s travel restrictions to combat COVID-19” by David Maher (Aug. 1st), piqued my interest. The study referenced is a wonderful work, combining an epidemiology model with machine learning to predict the effects of opening Newfoundland to the rest of the world. To get the details, I read the manuscript posted on medRxiv. Cautionary flags ensued.
MedRxiv is a preprint server. In academic publishing, a preprint is a version of a scholarly or scientific paper that precedes formal peer review and final journal publication. In fact, the medRxiv homepage explicitly states:
“Caution: Preprints are preliminary reports of work that have not been certified by peer review. They should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.”
My current role is editor-in-chief of an academic journal for Cell Press (Matter). I deal with such preprints on a daily basis. They aren’t bad. They are just an early version of a study — like the first cut of a film. The critical point is that these preprint manuscripts have not yet been peer reviewed.
Why is that a problem?
First, subject to peer review, most manuscripts undergo revisions. Some assumptions are asked to be justified, and more results are produced to support conclusions. Unlikely to change the primary findings of any study, there could be subtle differences between the preprint and the final published work. Reporting before revisions could ultimately report false information. Second, and of greater consequence, the paper could be rejected post-review. Perhaps there was a critical flaw or error that the authors had missed. Perhaps there is a major missing piece to the argument. It happens all the time.
In times when leaders should turn to science rather than politics to make policy and societal decisions, we want the underlying science to be as trustworthy as possible.
Of more subtle concern — amplified recently due to the COVID crisis — is the rush to announce scientific findings before they have been sufficiently vetted. Physicians and politicians are desperate for reliable and actionable information on COVID-19. There were early non-peer-reviewed studies on hydroxychloroquine, and I need not point out the resulting quagmire. While researchers and experts understand that the information posted on preprint servers may or may not be reliable, such data may be unintentionally misunderstood by the general public or distorted by those who post comments on social media. This is at best misleading, and at worst leads to (dare I say it) the emergence of fake news. While we should act quickly on COVID research, we have to ensure sure the science is still scrutinized and stands up to critique.
We have to trust the process.
With all that being said, I don’t mean to delegitimize the work of Dr. Proton Rahman and colleagues. It is neither misinterpreted nor misleading and is the result if renowned scientists from acclaimed institutes. It is just putting the cart before the horse, so to speak. Being in the materials science field, I am familiar with the works of both Prof. Ellen Kuhl and Prof. Alain Goriely. Moreover, while not familiar with Rahman’s works personally, I do not doubt his credentials. This is a rigorous study. It will likely be published in a reputable journal (the authors can reach out to me if they are interested in Cell Press). However, care should be taken before The Telegram, or any other media outlet, announces scientific findings with the implication they are definitive. Particularly prior to peer review.
In times when leaders should turn to science rather than politics to make policy and societal decisions, we want the underlying science to be as trustworthy as possible. At least as trustworthy as the free press. It just makes sense.
Steven W. Cranford, Ph.D.
Editor-in-Chief, Matter (Cell Press)