This article was my entry to the Better Science Through Better Data writing competition 2018. You can find out more here and read my report from the day here.

With the BBC reporting a “reproducibility crisis” in 2017 and a Nature survey concluding that more than 70% of researchers have failed to reproduce the results of previous research, it is more important than ever for journals and publishers to ensure the reproducibility of results before publication so that results are valid.

A study by Nicole A. Vasilevsky et al. found that only 11.9% of a sample of 318 journals explicitly stated that data sharing was required for publication. Consequently, researchers often only provide a subset of publishable results with any “unsuccessful” results ignored. This can then make it difficult for people reproducing research as they do not have access to all data used in the study. If publishers insisted that all data from research be shared before publishing papers then experiments would become more reproducible. Although an increase in available data would provide the tools with which to reproduce research, it is still difficult to encourage scientists to reproduce the work of others as opposed to working on new research.

Unfortunately, the emergence of a “publish or perish” mentality has increased the pressure placed on scientists to produce new and exciting results suitable for publication, resulting in a reduction of the reproduction of experiments. Publishers are partly responsible for this mentality as journals are dedicated to new research and rarely report on the reproduction of results. Researchers require publication to receive grants and the need for funding launches scientists to push research forward without spending time ensuring previous results are valid. Both grant providers and publishers should act on this. When awarding grants, grant providers should encourage discussions about research and consider the importance of replications by providing recognition for testing reproducibility. Publishers should create a dedicated section of their journals for reproduced research to encourage scientists to review one another’s work. Some journals are already taking steps towards this.

Nature Human Behaviour is one journal that is acting on the reproducibility dilemma. The registered report format of this journal means that a research paper is initially peer reviewed before data is collected and publishers decide whether to accept the paper at this stage. This shifts the focus of the publication to the quality of the research questions and the discussion of the results for better or worse instead of focusing only on “successes”. Results seen as “unsuccessful” can also include a lot of interesting information. Consequently, the researchers have no need to exaggerate or withhold results. With this method of publication, scientists are required to publish all raw data, materials and code. All these factors lead to the production of reproducible science and enable research to be replicated more easily by other scientists.

Journals and publishers have responsibilities as the distributers of research to monitor and encourage reproducibility amongst the scientific community and should insist on the publication of raw data and encourage discussions about research. Some journals are already considering this, but this practice is yet to be taken up by the majority.