An earlier pilot project found that the use of SciScore, an AI-based tool, to evaluate submitted manuscripts was effective in improving several aspects of rigor of research articles, prompting the expansion of SciScore to more titles.
The reproducibility crisis in scientific research is a many-sided problem involving ingrained practices, both in laboratory and publication practices. Fortunately, some of the root causes of irreproducibility are known and being addressed by scientists and editors alike through a steadily increasing number of recommendations and requirements used to address different aspects of rigor and reproducibility in the scientific literature, e.g., MDAR, ARRIVE, CONSORT, and RRID standards.
SciScore, a product of SciCrunch Inc., provides authors and reviewers with both a score and a supporting report of the submitted manuscript to identify if key areas of reproducibility and transparency have been addressed or not [MK1] in the manuscript. It will look for evidence of randomization, blinded conduct of the experiment, sample size estimation, whether sex is included as a biological characteristic, animal/cell line authentication or contamination, verify the identity of the antibodies used, and various other guideline-specific items.
SciScore, an automated and multifaceted tool based on artificial intelligence, evaluates scientific manuscripts for adherence to several key reporting criteria for rigor and reproducibility. To improve the reproducibility of the research published, SciScore is now integrated into the American Heart Association’s journal submission workflow. Results from the initial pilot, which ran over a year’s time for the Association’s Circulation Research journal, showed that the average monthly scores for submitted manuscripts rose dramatically by 4% month-over-month, which helped the journal to improve its rigor and reproducibility.
“SciScore helps editors point out the places where authors may have omitted a bit of detail about their study,” says Anita Bandrowski, CEO of SciScore. “I know that most researchers, when asked about methodological details, are happy to supply needed information. It is great that SciScore can alert authors and reviewers in advance where things might be missing. It is like spell check but for reagents.”
Moving forward, the following American Heart Association journals will have SciScore integrated in their journal submission process: Arteriosclerosis, Thrombosis, and Vascular Biology (ATVB), Circulation Research, Stroke, Hypertension, and Stroke: Vascular and Interventional Neurology.
Further reading on reproducibility crisis
- https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002165
- https://blogs.scientificamerican.com/observations/to-fix-the-reproducibility-crisis-rethink-how-we-do-experiments/
- https://www.ahajournals.org/doi/full/10.1161/STROKEAHA.108.515957
Further results with SciScore
