How do we measure success for Open Science?

To deliver the benefits of Open Science – economic, societal, research impact and for researcher careers – increasing adoption of Open Science practices is a prerequisite goal. There is a growing need for reliable data on Open Science practices to better understand researchers’ needs, and to understand the effectiveness of policies and solutions at increasing adoption of Open Science. In 2022 PLOS launched ‘Open Science Indicators’, (OSI) an initiative that tracks adoption of Open Science practices – such as sharing data, code, protocols and preprints – over time in the scholarly literature. This initiative aligns with UNESCO’s goal to implement an Open Science Monitoring Framework. PLOS developed requirements for an OSI measurement framework, underpinned by six guiding principles: (1) Align with established community definitions or approaches wherever possible; (2) Measure what practices are being carried out now; (3) Ensure interoperability across diverse communities; (4) Be scale-able across large volumes of research outputs; (5) Share results of Open Science Indicators/ monitoring activities openly; (6) Use Open Science Indicators responsibly. OSIs are being delivered in partnership with DataSeer, using a repeatable approach that can be delivered at scale across thousands of published articles, and which reaches recognised thresholds for accuracy. Importantly, OSIs should not be used to rank journals, institutions or individuals. The initial results of the project analysed nearly 70,000 articles published between the start of 2019 and the end of Q2 2022 and results are being updated through 2023 as new articles are published (https://theplosblog.plos.org/2022/12/open-science-indicators-first-dataset). The results show changes in adoption of data, code and preprint sharing, and reveal differences in adoption – and potential for adoption – between communities and regions, The data are being shared openly to engage the community in a conversation about how we can use better quantitative, longitudinal evidence on Open Science practices to support increased adoption of Open Science globally.

Watch it on Vimeo
Download the slides in Zenodo