Science policy Remote Presentation
14 Nov 2021 09:00 AM - 11:45 AM(America/New_York)
20211114T0900 20211114T1145 America/New_York Opening Up Open Science

The open science movement is attracting significant attention from the scientific community in an effort to increase the reliability of science, promote scientific innovation, and make scientific information widely available to non-specialists. It encompasses a wide variety of initiatives, such as publishing in open-access venues, making study data and methods publicly available, pre-registering study designs in trial registries, encouraging the publication of negative as well as positive results, and making peer review reports publicly available.

 

As this movement develops, philosophers of science interested in doing socially engaged work have important opportunities to help shape open science in ways that maximize its strengths and minimize its weaknesses. This symposium focuses especially on four aspects of the open science movement that merit philosophical consideration: the importance of fostering epistemic diversity and pluralistic confrontation within open science, the challenges of implementing pre-registration as a strategy for promoting replicability, the potential for using preprint repositories to certify research quality while promoting openness, and the steps needed to fulfill the open science movement's goal of facilitating better decision making by non-specialists.

PSA 2020/2021 office@philsci.org

The open science movement is attracting significant attention from the scientific community in an effort to increase the reliability of science, promote scientific innovation, and make scientific information widely available to non-specialists. It encompasses a wide variety of initiatives, such as publishing in open-access venues, making study data and methods publicly available, pre-registering study designs in trial registries, encouraging the publication of negative as well as positive results, and making peer review reports publicly available.

 

As this movement develops, philosophers of science interested in doing socially engaged work have important opportunities to help shape open science in ways that maximize its strengths and minimize its weaknesses. This symposium focuses especially on four aspects of the open science movement that merit philosophical consideration: the importance of fostering epistemic diversity and pluralistic confrontation within open science, the challenges of implementing pre-registration as a strategy for promoting replicability, the potential for using preprint repositories to certify research quality while promoting openness, and the steps needed to fulfill the open science movement's goal of facilitating better decision making by non-specialists.

Open Science and Epistemic Diversity: Friends or Foes?
Symposium Paper Abstracts 09:00 AM - 09:30 AM (America/New_York) 2021/11/14 14:00:00 UTC - 2021/11/14 14:30:00 UTC

The potential of Open Science [OS] to enhance research quality, reliability, integrity and societal impact has been widely discussed in academia and policy. The European Commission defined OS as "a new approach to the scientific process based on cooperative work and new ways of diffusing knowledge by using digital technologies and new collaborative tools". This runs counter historical and philosophical work that attest to the enduring significance of critical scrutiny, collaboration and sharing of data, methods and materials as constitutive of scientific practice. Despite vast efforts to implement OS over the last decade, it also remains unclear how this vision relates to the widespread diversity in epistemic practices favoured by different research communities. OS appears to require the adoption of common metrics, principles, standards and platforms, which threaten to privilege some theoretical perspectives – and related ways of knowing - over others, thus disrupting well-established methodologies and creating divides within and across research domains. At the same time, the OS movement represents an opportunity to redesign research governance to better account for the diversity in research environments in which researchers around the globe operate, as well as the value of the diverse outputs produced in the course of inquiry - including data, models, software, techniques, instruments and samples. I will argue that OS risks acting as a reactionary force which reinforces conservatism, discrimination, commodification and inequality in research. Research on the impact of Open Data policies shows how the rush to put research data online may have dire consequences, such as contributing to unregulated surveillance of individuals and communities (Taylor et al 2017); expanding existing divides and silencing knowledge from low-resourced environments (Bezuidenhout et al 2017); and privileging the re-use of existing data over the creation of data tailored to the research at hand (Leonelli 2017, 2020). These dangers can be averted by ensuring that OS acknowledges and promotes epistemic diversity, for example by: challenging traditional communication channels and entrenched power structures within academia; and fostering a pluralist ecosystem of perspectives, methods and norms that is explicitly tailored to the situation, goals and resources of researchers. The argument builds on the extensive philosophical literature on epistemic pluralism; experience as OS advisor to the European Commission; and empirical studies of OS practices (e.g. Leonelli 2013, Levin and Leonelli 2017).


Bezuidenhout, L., Leonelli, S., Kelly, A., Rappert, B (2017) Beyond the Digital Divide: Towards a Situated Approach to Open Data. Science and Public Policy 44(4): 464-475. 

Taylor, L, Floridi, L. and Bart van der Sloort (eds) (2017) Group Privacy: New Challenges of Data Technologies. Springer. 

Leonelli, S. (2020) Learning from Data Journeys. In: Data Journeys in the Sciences. Springer.

Leonelli, S. (2017) Global Data Quality Assessment and the Situated Nature of "Best" Research Practices in Biology. Data Science Journal 16(32): 1-11.

Leonelli, S. (2013) Why the Current Insistence on Open Access to Scientific Data? Bulletin of Science, Technology and Society 33(1/2): 6-11. 

Levin, N., Leonelli, S. (2016) How Does One "Open" Science? Science, Technology and Human Values 42 (2): 280-305. 

Presenters
SL
Sabina Leonelli
University Of Exeter
Openness About Openness: Why Pre-registration Is Failing and What To Do About It
Symposium Paper Abstracts 09:30 AM - 10:00 AM (America/New_York) 2021/11/14 14:30:00 UTC - 2021/11/14 15:00:00 UTC

The open science movement has defended pre-registration enthusiastically to address the replicability crisis in the social and behavioral sciences (Nosek et al., 2019). A pre-registration is a timestamped and uneditable plan that states the hypotheses and experimental protocol of a study. In theory, pre-registration increases the reliability of findings by reducing "researcher degrees of freedom", e.g., it prevents questionable research practices (e.g., p-hacking) and post-hoc hypothesizing.

Despite the enthusiasm, recent meta-scientific studies reveal that pre-registration has not delivered its epistemic benefits in practice (Veldkamp et al., 2018; Claesen et al., 2019).

In this talk, I argue that pre-registration fails because the incentives to do rigorous pre-registration do not meet the epistemic requirements of the practice. I show this failure in the two current pre-registration models:

Plain Pre-Registration. In this model, pre-registration is optional. As such, it does not guarantee that findings will be reported and hence, does not counter publication bias (Chen et al., 2016). Moreover, the researcher who pre-registers incurs costs that put her in a competitive disadvantage (e.g., being scooped.)Registered Reports (Chambers, 2013). In this model, scientists submit a research proposal to a journal that can be accepted before data collection. This means that the paper will be published regardless of its outcome. While this model provides incentives to pre-register, many journals that have incorporated registered reports have decided to keep the registered protocols private (Hardwicke & Ioannidis, 2018).

In response, I propose a third model for the social and behavioral sciences: mandatory open pre-registration for confirmatory projects. In this model, researchers make pre-registration a standard requirement independently of the publication system. While the replicability crisis requires more interventions (Romero, 2019), this model approaches the epistemic goals of pre-registration and reduces potential competitive disadvantages. I suggest that embedding open-science practices into the traditional publication systems and incentive structures compromises their openness.

References 

Chambers, C. D. (2013). Registered reports: A new publishing initiative at cortex. Cortex: A Journal Devoted to the Study of the Nervous System and Behavior, 49(3), 609 - 610. 

Chen, R., Desai, N. R., Ross, J. S., Zhang, W., Chau, K. H., Wayda, B.,…, Krumholz, H. M. (2016). Publication and reporting of clinical trial results: cross sectional analysis across academic medical centers. BMJ, 352

Claesen, A., Gomes, S. L. B. T., Tuerlinckx, F., & Vanpaemel, W. (2019). Pre-registration: Comparing dream to reality. PsyArXiv. 

Hardwicke, T. E., & Ioannidis, J. P. A. (2018). Mapping the universe of registered reports. Nature Human Behaviour, 2(11), 793–796. 

Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The pre-registration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. 

Romero, F. (2019). Philosophy of Science and The Replicability Crisis. Philosophy Compass

Veldkamp, C. L. S., Bakker, M., van Assen, M. A. L. M., Crompvoets, E. A. V., Ong, H. H., Nosek, B. A.,…, Wicherts, J. M. (2018). Ensuring the quality and specificity of pre-registrations. PsyArXiv. 

Presenters
FR
Felipe Romero
University Of Groningen
Certify Pre-Prints
Symposium Paper Abstracts 10:00 AM - 10:30 AM (America/New_York) 2021/11/14 15:00:00 UTC - 2021/11/14 15:30:00 UTC

Sociologists and philosophers of science have long recognized that scientific knowledge must be "socially shared and socially validated" (Merton 1973, 59), where peer reviewed journal publication has long served as "the standard avenue" for playing the dual role of sharing (Longino 2002, 76) and certifying knowledge (Zuckerman and Merton 1971). Although journal publication was once thought to be an act of giving one's work "to the common fund of knowledge" (Merton 1973, 59), journals have increasingly consolidated under the umbrella of a few monopolistic publishers who charge public universities exorbitant subscription fees, with profit margins that best big pharma and tech (Buranyi 2017). Activism around the cost and inaccessibility of scientific knowledge has increased sharply: a coalition of national research funding organizations will require that publicly funded articles be published open access effective 2021 (Plan S); and, the UC system announced that it would drop its ~$11 million annual subscription to Elsevier.

How can authors in the sciences protect scientific knowledge as a public good while also certifying the quality of their work? This talk will discuss how meta-science as well as social and technological shifts have paved the road towards creating new systems for 

certifying publicly shared science that moves beyond traditional peer reviewed journals, including open access journals. Building on previous work suggesting that we deconstruct journals into their purpose-oriented components (Smith 1999) and calls to leave pre- publication peer review behind (Heesen and Bright 2019), I will take a functional and implementational view to outline capacities that preprint repositories can integrate into their platforms to permit certification of the quality, as well as the transparency and openness, of archived papers (Aalbersberg et al 2018) while avoiding the pitfalls of earlier innovation attempts (Sugimoto 2017).

References

Aalbersberg, IJsbrand Jan, Tom Appleyard, Sarah Brookhart, Todd Carpenter, Michael Clarke, Stephen Curry, Josh Dahl et al. 2018. "Making Science Transparent by Default; Introducing the TOP Statement."

Buranyi, Stephen. 2017. "Is the Staggeringly Profitable Business of Scientific Publishing Bad for Science?" The Guardian.

Heesen, Remco, and Liam Kofi Bright. 2019. "Is Peer Review a Good Idea?" British Journal for the Philosophy of Science.

Longino, Helen E. 2002. The Fate of Knowledge. Princeton: Princeton University Press. Merton, Robert K. 1973 "The Normative Structure of Science." In The Sociology of

Science, ed. Norman Storer, 267-78. Chicago: University of Chicago Press.

Piwowar, Heather, Jason Priem, Vincent Larivière, Juan Pablo Alperin, Lisa Matthias, Bree Norlander, Ashley Farley, Jevin West, and Stefanie Haustein. 2018. "The State of OA: A Large-Scale Analysis of the Prevalence and Impact of Open Access Articles." PeerJ 6:e4375.

Smith, John WT. 1999. "The Deconstructed Journal-a New Model for Academic Publishing." Learned Publishing 12:79-91.

Sugimoto,CassidyR.,SamWork,VincentLarivière,andStefanieHaustein. 2017. "Scholarly Use of Social Media and Altmetrics: A Review of the Literature." Journal of the Association for Information Science and Technology 68:2037-62.

Zuckerman, Harriet, and Robert K. Merton. 1971. "Patterns of Evaluation in Science: Institutionalisation, Structure and Functions of the Referee System." Minerva 9:66- 100.

Presenters
CL
Carole Lee
University Of Washington
Making Open Science Work for Non-Specialists
Symposium Paper Abstracts 10:45 AM - 11:15 AM (America/New_York) 2021/11/14 15:45:00 UTC - 2021/11/14 16:15:00 UTC

One of the major justifications for the open science movement is to make usable information available to non-specialists. For example, the U.S. National Academy of Sciences states, "The belief that the broader public should have access to publicly-funded research and its benefits provides an additional strong rationale for open science" (NAS 2018, 2). The British Royal Society also notes that "many of us want to satisfy ourselves as to the credibility of scientific conclusions that may affect our lives, often by scrutinising the underlying evidence" (Royal Society 2012, 7), and thus the Royal Society affirms that when scientific data are particularly relevant to the public, they should be "accessible, intelligible, assessable and usable for the likely purposes of non-specialists" (2012, 8). 

Unfortunately, while the goal of equipping non-specialists to make better decisions in response to scientific information is an important one, most of the major initiatives associated with the open science movement are not well placed to achieve this goal. Most members of the public are not able to evaluate or make use of the data released by scientists, and open-access publications are comprehensible only to a very small fraction of the population. Thus, further reflection is needed about how to leverage the activities of the open science movement so that they can be as useful as possible for public decision-making. 

This paper examines recent controversies about Lyme Disease as a case study for exploring how the open science movement can better achieve its goal of making usable information available to non-specialists. Analysis of this case illustrates that meaningful openness will require not only efforts to provide open access to data and publications but also efforts to create a network of individuals and institutions who can help interpret and analyze this information in ways that meet the goals of different stakeholders. For most non-specialists, it is less important to have direct access to data and publications regarding Lyme Disease than to have information about the crucial value judgments that affect how different experts interpret and apply the available research. Moreover, various stakeholders differ in terms of the value judgments that matter to them and the level of explanation they need in order to make sense of those judgments. Thus, achieving the goals of the open science movement requires not only openness from the scientists involved in Lyme Disease research but also strategic steps taken by scholarly societies, government agencies, physicians, journalists, and patient advocacy groups. Thus, this case illustrates how philosophers of science can make a valuable contribution to the open science movement both by clarifying what is needed to fulfill the goals of the movement and by examining how the activities of multiple groups can help contribute to meeting those needs. 

References

NAS (National Academies of Sciences, Engineering, and Medicine) (2018). Open Science by Design: Realizing a Vision for 21st Century Research. Washington, DC: National Academies Press.

Royal Society. (2012). Science as an Open Enterprise. London: The Royal Society.

Presenters Kevin Elliott
Michigan State University
Commentary
Symposium Paper Abstracts 11:15 AM - 11:45 AM (America/New_York) 2021/11/14 16:15:00 UTC - 2021/11/14 16:45:00 UTC

Commentary

Presenters
PS
Patricia Soranno
Michigan State University
Michigan State University
University of Washington
University of Groningen
University of Exeter
Michigan State University
University of Western Ontario
 Cristian Larroulet Philippi
University of Cambridge
The University of Western Ontario
Ecoles des hautes études en sciences sociales (EHESS)
University of Calgary
 Inkeri Koskinen
Tampere University
+17 more attendees. View All

Notes

No notes added.
Program Navigator