OAI12 Session 3
Changing times, challenging norms: How are changes in research practice re-shaping our thinking about what research integrity should be?
Often the language around ‘research integrity’ is focused on the behaviour of the individual researcher. Yet issues with research reproducibility and replicability are systemic. By re-focusing on the integrity of the research itself, rather than relying on ‘a few good people’, many of the issues will, if not completely disappear, be considerably less likely to occur. New institutional approaches to fostering reproducibility and research integrity are being developed. National research integrity networks in countries like Switzerland and the United Kingdom have been launched, and have generated initiatives like the Centre for Academic Research Quality and Improvement at Bristol University. Meanwhile, challenges to existing assumptions about research integrity continue to emerge. These are technological, as in the development of innovations which rely on opaque machine learning models or cultural and political such as evaluation frameworks which encourage ‘gaming’ in ways which dis-incentivise good practices and transparency. These pose new challenges to the already-problematic notion of ‘universal norms’ for science and research in a world of proliferating technology and every wider-networks of communication and collaboration.
This session brings together research practitioners who are grappling with these emerging challenges, and members of new institutions created to respond to them. We will learn about their work, and about the tensions and uncertainties which are changing the way we approach reproducibility and research integrity. The session will centre on discussion and debate, in which we will explore questions like: How are changes in research practice re-shaping our thinking about what research integrity should be? Does a focus on individual careers cause us to lose sight of the fact that research is a common endeavour? Can a broad definition of ‘openness’ help us to understand the myriad ways that ‘research integrity’ is evolving across disciplines and cultural groups? This will be a lively, informative and interesting session!
Wednesday 08 Sept. 2021
Incentivising sustainable and collaborative research
Working in a truly open, reproducible and inclusive way is not yet fully supported within modern research culture. Metrics, rewards and progression still focus on the publication record of the individual researcher and the end product of the research, instead of the process by which the research is conducted. For example, the sustainability of software and data which underpin research is vital to ensure the work is reproducible in the future and can be built upon, but is not sufficiently incentivised or supported. Emerging roles and ways of working highlight how unfit current measures of success in research are. Systemic solutions such as promoting collaborative ways of working and professionalising alternative but essential roles to support them, such as Research Software Engineers, Data Stewards and Community Managers, can lead to more sustainable, reproducible and efficient research.
Implementing Systems-level Reform: Institutional Change towards Transparency
In recent years, the academic community has evaluated the research ecosystem and identified key issues which undermine the trustworthiness of its output. With it, myriad suggestions, and solutions. Despite this, change is slow and well-meaning initiatives often have adverse reactions. This is because the process of determining a vision for an ideal research system and implementing it are altogether different challenges. Furthermore, research systems in Asia, Latin America, and Africa are substantially more heterogenous than in North America, Europe, and Australia. For example, in many countries in the Global South, research culture is still labile due to the sudden introduction of policies and unique incentive structures which champion research quantity .Thus many ideas generated in one research system do not translate well and vice versa. The question becomes given reach region's unique state, how do we champion process and institutional transparency? In this talk I discuss the different factors that affect how we approach each country, including but not limited to existing policies, centralisation of research authority, and inherent cultural beliefs. Further, I outline strategies for reform using Indonesia as an example, from a grassroots movement to influencing national infrastructure and policy.
Moving from Trust to Trustworthiness
Academia represents a paradox. On the one hand, many of the methods and techniques used across disciplines are cutting edge and constantly evolving. However, at the same time, our underlying cultures and working practices remain rooted in the 19th Century model of the independent researcher. Research groups are effectively small, artisanal businesses, each crafting outputs – often exquisite, but the product of the unique skills and processes of that group. This model risks poor reproducibility and replicability of research – closed workflows, closed data, use of proprietary file formats, and variability in skills across researchers. It also relies on trust – we have to trust that the researcher or group has fully disclosed all aspects of the process that generated an output. There is empirical evidence that this is not the case, a situation exacerbated by current incentive structures but also legacy systems such as journal article limits on word counts and display items – an echo of the print medium that is now largely defunct. However, existing technologies allow us to make transparent many (if not all) of the elements of a research workflow – protocols, data, code and so on. This transparency of process has the potential to make the system inherently more trustworthy, by allowing it to be scrutinised, thereby moving away from a model of trust in individuals.
Coffee break and Music
How trustworthy is ’science'?
During the past year, ’trusting the science’ and ‘following the science’ have become top of every news agenda as it has become abundantly clear that the pronouncements of scientists can mean life or death for millions of people. In reality, the outputs of the research community dramatically affect all our lives whether there is a pandemic or not: published research forms the basis of our medical treatments, economic decisions, development programmes, conservation initiatives and much, much more. But does academic research provide a firm enough base on which to build our world? This is the real definition of ‘research integrity’ - research that you can trust with your life. I venture to suggest that researchers in most disciplines would not bet their life on the predictions of their own field. To produce research with this level of integrity requires an approach different from that found in many disciplines today: a professionalisation of the research community, a clarification of goals and re-alignment of incentives, and the development of tools to match those goals and professional pathways.
Most young scientists are excited by the prospect of helping solve the world’s problems and ‘making a difference’. During their PhDs and onwards, most are taught instead that their aim should be to write scientific papers, the great majority of which will have minimal readerships and real-world impact. If we want our research base and our science to be truly trustworthy we need instead to continue to inspire and enable bright minds to come up with ideas, to test them exhaustively and work out how they can be used in the real world. We need sparks of creativity to meet relentless slog, sharp critique to meet appreciation and encouragement, practical experience to meet fresh perspectives. In short: a very different research culture. And I think that can be achieved through a relatively simple change in the way that research is shared.