eLife

A project to assess the reproducibility of biomedical research in Brazil has been described today in the open-access journal eLife.

The Brazilian Reproducibility Initiative is a systematic, multi-centre effort to repeat experiments from published articles originating from Brazil, with studies due to begin later this year and results expected in 2021. It is the first project of its kind designed to estimate the reproducibility of research in a specific country.

Launched in 2018, the initiative was recently awarded a grant of R$1m (about US$275,000) from the Serrapilheira Institute – Brazil’s first private institute dedicated to supporting scientific research – to repeat between 60 and 100 experiments in at least three different laboratories each.

The replication studies will be based on individual experiments, rather than entire research articles, to increase coverage of the published literature and facilitate analysis. The scope of the studies will also be limited to common methodologies that are widely available in Brazil, to ensure they can be replicated within the project’s network of more than 70 collaborating laboratories.

If the original result of an experiment is within the variation range predicted by the replication studies, it will be deemed reproducible. The results will then be used to assess if there are features of the original articles that can predict reproducibility.

Olavo Amaral, Professor at the Institute of Medical Biochemistry, Federal University of Rio de Janeiro, Brazil, will lead the project, alongside a coordinating team that includes researchers Kleber Neves, Clarissa Carneiro and Ana Paula Sampaio, from the same university. They will be responsible for selecting the methods and experiments to replicate, as well as overseeing the work of the collaborating labs.

“Most efforts to estimate the reproducibility of published findings have focused on specific areas of research, such as psychology and cancer biology, even though science is usually assessed and funded on a regional or national basis,” Amaral says. “Here, we offer a different perspective on the concept, covering various areas of life science research with a focus on a particular country.”

Amaral adds that the project is not without its challenges. For example, there may be concerns that replicating other scientists’ work indicates mistrust of the original results. Also, as a degree of irreproducibility is expected – based on results from previous reproducibility efforts – this may raise worries about Brazilian science as there will be no estimates from other countries for comparison.

“To help counter these concerns, we are taking steps to ensure the project is viewed as we ourselves conceive it: a first-person initiative of the Brazilian scientific community to evaluate its own practices in an open, unbiased and transparent way,” Amaral says.

“Whatever the results may be, this project is bound to put Brazil at the vanguard of the reproducibility debate, if only because we will likely be the first country to undertake such an evaluation. Indeed, we hope other countries might be inspired to openly assess their own rates of reproducibility. This could help to guide national funding agencies in evaluating this central dimension of science, giving them insights on where to direct their investments and fostering the reliability of published research.”

The Feature Article ‘Science Forum: The Brazilian Reproducibility Initiative’ can be freely accessed online at https://doi.org/10.7554/eLife.41602. Contents, including text, figures and data, are free to reuse under a CC BY 4.0 license.