The open access journal, BMC Psychology, has announced it will launch the first ever randomized controlled trial to find out if a ‘results free’ peer-review process can help reduce publication bias.
‘Results free’ means that reviewers of research manuscripts submitted for publication will not be able to see the results or discussion sections until the end of the review process. It is thought that this could ensure the research is judged on the strength of a study’s methods, and the question it is addressing, rather than the results or outcome of the study.
Dr Katherine Button, from the University of Bath and advocate for improving the transparency of research, said: “Publication bias is a serious issue in academic publishing because it distorts the evidence available to clinicians, researchers and policymakers – potentially leading to false conclusions that could have severe consequences.”
It is well established that results deemed statistically significant are more likely to be published than null results – or those that fail to reach significance in a statistical test. However, these null results form an important part of the scientific record and are crucial to develop an accurate evidence base.
Dr Button said: “The current system favors publication bias because significant results are seen as more important to the scientific record by publishers, academics and the systems in place to measure their performance. This new trial should at least begin to address one area where publication bias arises.”
The trial will start with an initial pilot phase where the first 10 authors to opt in will go through the ‘results free’ process to show that that the procedure is feasible and efficient. Following the pilot, a randomized controlled trial will start where authors who opt in are randomly assigned to the ‘results free’ or normal peer-review process.
Liz Bal, Associate Publisher at BioMed Central, which publishes BMC Psychology, said: “Authors who are in the ‘results free’ group will submit a full manuscript for review but the results, discussion and conclusions will be hidden from the peer-reviewers. In the first stage of review, the manuscript will be ‘accepted in principle’ based on the rationale and methods only. These manuscripts will then be reviewed again by the same reviewers but with the omitted sections visible. At that stage, the decision to publish can only be revoked if the results and discussion deviate unjustifiably from the stated aims and methods. We believe that this could help reduce publication bias by basing the decision to publish purely on the scientific rigor of the study design.”
Evidence shows that unreliable research findings can be found across the biomedical and social sciences. Innovative reforms of the peer-review system, such as the ‘results free’ trial, could help address these issues.
