COVID-19 and the Acceptance of Preprints in Health Sciences Publishing

Dustin Sullivan, MPS

Abstract

Background

A “preprint” is a scholarly or scientific manuscript shared online by its authors in its pre-peer reviewed form. The paper is posted on a preprint server before other researchers have had a chance to review its methods and claims through peer review. Preprints are posted by the authors before, or sometimes simultaneously with, submission to traditional peer-reviewed scholarly journals. Preprints have gained increased visibility and influence in life and health sciences publishing during the COVID-19 pandemic, and there are signs that life and health sciences publishing is approaching a tipping point in preprints’ acceptance among healthcare workers. Is life and health sciences publishing seeing an evolution of attitudes regarding the use of preprints as a result of COVID-19, or will preprints remain the domain of the hard and social sciences?

Purpose

This paper examines the history of preprints in scientific literature, including the differing use of preprints by discipline and the ethical challenges presented by their acceptance. The results of a survey of healthcare workers, in which attitudes toward preprints and the influence COVID-19 has had on healthcare workers’ opinions were measured, are presented.

Methods

A cross‐sectional survey was conducted using a web‐based survey tool. This article also draws upon the author’s experience in the publishing industry. The author’s recommendations are based on both the survey results and personal experience.

Findings

Of the respondents, 65% disliked preprints, and 35% were in favor of their use. Notably, the favorability of preprints was positively influenced by the COVID-19 pandemic among those already accepting of preprints; the pandemic has increased their belief in the value of preprints. Consumers of healthcare research were more open to preprints than the researchers themselves.

Conclusions

Although preprints do not seem set to become standard practice in scholarly life sciences and healthcare publishing, clearly there are signs of their increased use and acceptance: the growth of preprint servers in the past few years, the number of survey respondents indicating more acceptance of preprints due to COVID-19, and the dramatic increase in preprint submissions starting in the first half of 2020—all indicating preprints’ growing relevance. Publishers can learn from the rapid developments of the past few years and take steps to make preprints, as well as traditional journal publishing, work for the benefit of all.

Author Bio

Dustin Sullivan, GW Publishing Alumnus, 2020, is publisher at Sigma Theta Tau International Honor Society of Nursing, where he manages Sigma’s peer-reviewed journals as well as their professional and academic book publishing program. Over the last 27 years in publishing, he has held many roles including author, technical editor, rights manager, acquisitions editor, and publisher. Prior to joining Sigma he was a senior acquisitions editor at Pearson Education.

1. Introduction

Traditional medical journals can take months to publish accepted manuscripts, requiring time to obtain peer review (from multiple reviewers), approve, edit, and publish articles. There was pressure to speed medical research to publication as the spread of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) reached pandemic status in early 2020. As the disease caused by the novel coronavirus, COVID-19, affected people around the world and befuddled healthcare workers, the need for information about the virus and how to best treat infected patients was paramount. As emergency departments and intensive care units filled with patients ill with COVID-19, there was great urgency to radically speed up the article publishing process or bypass some of the traditional checks built into this process.

In late February of 2020, as publisher of two peer-reviewed nursing journals, I started hearing from nurse researchers who were pleading for me to skip through the normal publication process and post their manuscripts right away. Some journals hastily developed streamlined approval and review timelines to answer the frantic call by researchers to make their work available as soon as possible, leading to some successes but also some very public failures (a few of which will be discussed below). Other researchers looked to another option for their papers, a process in use for decades in hard sciences and social sciences but only in its early stages for medical research: preprints.

Under normal circumstances, as a publisher, I would not hear directly from these nurse researchers. While nurse researchers often contact journal editors to discuss their research, they normally do not push, particularly the publisher, for their manuscripts to be fast-tracked or to skip peer review entirely. My prior experience working with journal editors and nurse researchers, as well as other publishing professionals, led me to believe the medical community (or at least nurse researchers) were not in favor of using preprints and preferred the normal peer review process. I was, however, aware of a small but growing segment in the medical community that was starting to experiment with preprints. Would the COVID-19 pandemic accelerate this growth and foster greater acceptance of preprints in healthcare? Were the emails from these nurse researchers demonstrating a change in beliefs brought about by the pandemic or merely indicating fear of a new viral foe?

I developed a brief survey for nurses and other healthcare workers to determine if opinions towards preprints were changing in the medical community—and if COVID-19 influenced these opinions. I made the survey open to all healthcare workers, not just the researchers who would be writing these papers, in an attempt to measure both the supply (researcher) and demand (article consumer) sides of the manuscript equation.

This paper provides some background on preprints, including some recent controversies; outlines some of the ethical challenges inherent in posting unreviewed or unverified research; presents the findings of my survey of healthcare workers; and attempts to draw some conclusions for medical researchers and publishers.

2. Definition and History of Preprints

Origins of Preprints

Preprints originated after World War II as physics researchers shared copied manuscripts via sneakernet (i.e., “walking a copy across the hall”) and other low-tech methods of distribution that included the postal service (Garisto 2019). Perhaps the most straightforward definition of a “preprint” is from the Committee on Publication Ethics (COPE), which defines it as “a scholarly manuscript posted by the author(s) in an openly accessible platform, usually before or in parallel with the peer review process” (COPE Council 2018, 2). In even simpler terms, a preprint is a scholarly or scientific manuscript shared by its authors before other researchers have had a chance to verify the veracity of its claims or the quality of its underlying research and methodology through peer review (a process designed to ensure quality and prevent bad manuscripts from being accepted). In today’s online world, the “openly accessible platforms” are websites and databases referred to as “preprint servers,” and preprints are posted by the authors before, or sometimes simultaneously with, submission to traditional peer-reviewed scholarly journals.

In addition to speeding up the initial posting process, submitting manuscripts to preprint servers before submitting them to traditional journals (or submitting to both simultaneously) offers several advantages to researchers and consumers of scientific papers, according to Kimberly S. Thompson, MLS, Ruth Lilly e-Repository Manager at Sigma Theta Tau International (Howard 2019, 6:40-8:00). According to Thompson, preprints have the following benefits:

• They share the research with a potentially large number of people in a highly visible manner.
• They facilitate broad (and often public) discussion with the manuscript authors.
• They lead to revision and possibly collaboration with other research teams.

These statements are echoed by the European educational think tank Knowledge Exchange: “Early and fast dissemination, increased opportunities for feedback and openness are seen as the main benefits of preprints” (Chiarelli et al. 2019, 7).

Use in Disciplines

When the first online preprint server, arXiv (pronounced like “archive”), was launched in 1991, “paper preprints had been entrenched in the culture of physics for decades.” Still, even as recently as October 2019, only about two percent of the 30 million peer-reviewed articles in PubMed, a search engine of life sciences and biomedical content maintained by the National Library of Medicine and the National Institutes of Health, were originally disseminated as preprints (Garisto 2019).

As noted, physics was the first discipline to take advantage of preprints, followed in large numbers by other physical sciences (e.g., chemistry, engineering) as well as social sciences (e.g., economics, history). Preprint server examples found today include arXiv (physics), SocArXiv (sociology), RePEc (economics), ChemRxiv (chemistry), engrXiv (engineering), LawArXiv (law), and PaleorXiv (Paleontology), and bioRxiv (biology). 

Currently there are more than 60 preprint servers available around the world, and life sciences and health have made more use of preprints in recent years. In a period from 2016 to 2019 referred to as the “second wave” of preprints, more preprint servers were started than had even existed in the previous 25 years (Johnson and Chiarelli 2019). This second wave included more medical and nursing research topics, including perhaps most notably medRxiv for health sciences, launched in mid-2019.

3. Ethical Challenges for Health Sciences Preprints

The phrase, “First, do no harm,” despite popular colloquial attribution, is not in the Hippocratic Oath, nor is it present in any standards or code of ethics held by the medical profession. (It might also surprise some to learn nursing students do not recite that oath at graduation, although some nurses do recite a “Nightingale Pledge” at graduation that is loosely based on the Hippocratic Oath.) One translation of the Hippocratic Oath does have something close, however: “I will follow that system of regimen which, according to my ability and judgment, I consider for the benefit of my patients, and abstain from whatever is deleterious and mischievous” (Shmerling 2015).

Despite the absence of this exact sentiment in the ancient oath, elements of that thinking persist in modern oaths, such as the Declaration of Geneva, the Canons of the American Medical Association, and the International Code of Medical Ethics (Annas 2017). The International Code of Medical Ethics almost seems to speak to the existence of preprints in one section, stating, “A physician shall recognize his/her important role in educating the public but should use due caution in divulging discoveries or new techniques or treatment through non-professional channels” (World Medical Association 2018).

But are preprints the type of “non-professional channels” this code is referring to? And again, what about times of pandemic? Can the COVID-19 pandemic be likened to wartime, when the urgency of the moment requires more risk-taking and less aversion to failure? Harvard professor Robert H. Shmerling, MD, provides a practical interpretation: “The fact is that when difficult, real-time decisions must be made, it’s hard to apply the ‘first, do no harm’ dictum because estimates of risk and benefit are so uncertain and prone to error. But it is a reminder that we need high-quality research to help us better understand the balance of risk and benefit for the tests and treatments we recommend” (Shmerling 2015).

Potential for Harm

Despite the perceived advantages to preprints as noted earlier and the rush to launch new preprint servers over the past four years, some fundamental questions remain about preprints in the life sciences and health disciplines that do not usually apply to the physical and social sciences. As summarized by Knowledge Exchange, “The main concerns over preprints are the lack of quality assurance, media potentially reporting inaccurate research and journals rejecting articles if a preprint has been posted” (Chiarelli et al. 2019, 7). Ignoring the possibility of journal rejection (which is an editorial, not ethical, decision), the concern can be summed in the following equation: no peer review plus wide dissemination plus misinterpretation by the media equals incorrect (and potentially harmful) information shared with the public.

Ethically speaking, is it wise to have open discussions and reporting on early research when there is a possibility of harm caused by sharing information that has not been peer reviewed for safety or even effectiveness (Tierney 2019)? To put it in more practical terms, if an astronomy preprint has the location of a protoplanet slightly wrong, other research may be affected, but otherwise, there are few (if any) dangers to the general public. However, if a preprint describing a pharmaceutical treatment for COVID-19 includes incorrect dosing, patients could suffer or even die.

Prior to the COVID-19 pandemic, the journal Clinical Orthopaedics and Related Research summarized the risks rather directly: 

We believe that the benefits proposed by advocates of medical preprint servers can be better achieved in other ways, and that medical preprint servers pose serious health and safety dangers to the patients for whom [we] are supposed to be caring…  Preprint servers may be perceived by some (and used by less-scrupulous investigators) as evidence even though the studies have not gone through peer review; the public may not be able to discern an unreviewed preprint from a seminal article in a leading journal… For the vast majority of medical (and especially surgical) diagnoses, a few months of review of a study’s findings do not make a difference; the pace of discovery and dissemination generally is adequate. (Leopold et al. 2019)

Publicity and Media Attention

During pandemic times, there are many healthcare providers who would argue that the typical “pace of discovery and dissemination” is not at all adequate. However, the additional media interest in medical topics during a pandemic increases the stakes and could amplify the impact of highly-publicized research errors (Bastian 2020). Poorly designed studies and otherwise bad research could “go viral” and obscure good research. Unreviewed preprints could end up misleading millions of readers, even if the authors themselves never intended their results to be widely publicized before publication (and review) in a traditional medical journal (Sheldon 2018).

While my intent is not to demonize journalists, the COVID-19 pandemic has produced multiple examples of studies that have been widely publicized on news websites and social media prior to receiving scientific scrutiny. A preprint posted early in the pandemic, for example, claimed to have found commonalities between the genetic structure of the novel coronavirus and the HIV virus (Pradhan et al. 2020). Although the paper was retracted, the retraction did not come with nearly the same amount of publicity as the initial paper.

In the most egregious examples, results have simply been misinterpreted by reporters, leading to some outrageous conclusions. The Economist published an article suggesting that a recent study concluded that smoking may have benefits that protect patients against COVID-19 (The Economist 2020). While absurd on the surface, the article detailed how statistics in the research showed something in cigarettes (nicotine?) led to smokers having lower rates of death than non-smokers. This was disproved by further scrutiny by a researcher experienced in assessing causality in observational data, but his counterpoint was published on a website that receives far less traffic than The Economist (Wilson 2020).

Additionally, several newspapers published headlines full of optimism related to work on a potential COVID-19 vaccine. The Washington Post (Johnson 2020) declared “Moderna’s coronavirus vaccine shows encouraging early results,” making the article one of their most widely-read articles on the day of its posting. The “early results” referred to in the headline were not from any published studies, however, let alone a manuscript posted to a preprint server. They were merely included in a corporate press release that failed to provide any of the underlying data that led to the rosy assessment. While preprints certainly cannot be blamed for this, this situation shows the eagerness of reporters to find and share information regardless of its accuracy.

These problems have been noted and in at least one case brought to light by government officials. In Australia, Chief Scientist Dr. Alan Finkel created the Rapid Research Information Forum with the Australian Academy of Science to ensure that questions about COVID-19 are answered by experts who can interpret the data and keep bad science from circulating widely (Davey 2020). In a sense, they created a government-run peer-review service.

Peer Review to the Rescue?

Scholarly publishing has a time-tested (albeit imperfect) way to ensure quality and prevent bad manuscripts from publication: peer review. Peer review is a fundamental process for both commercial and open-access journal publishing. The COPE Council (2017) notes, “Peer reviewers play a role in ensuring the integrity of the scholarly record” (2), while the Association of University Presses (AUP) remarks that peer review “provides feedback that is both stringent and fair, enables an author to strengthen a work in progress, and adds value and meaning to the work that is ultimately published.” AUP has even developed a detailed list of peer review best practices for member organizations (AU Presses Acquisitions Editorial Committee 2016), and many publishers have developed their own stringent peer review guidelines.

Peer review can prevent bad research from being published as well as point out fixable mistakes in otherwise good research. As noted earlier, in health sciences publishing, a bad study could lead to injury or death for patients, But some see a more practical side of peer review: preventing wasted time. As noted by Journal of Nursing Scholarship editor Susan Gennaro, PhD, RN, FAAN, implementing the science behind studies and research requires effort and time, and bad science can waste a lot of time and affect good patient care, even if no people are put in mortal danger. As Gennaro remarked in an interview about preprints, “To spend time doing an intervention that doesn’t help… hurts” (Howard 2019, 38:34-38:41).

Publishers’ Reactions to COVID-19

With healthcare providers scrambling for new treatments for COVID-19 patients and researchers dropping everything to study the novel coronavirus and COVID-19, publishers have tried to adapt and find ways to speed articles to publication. Publishers started with areas they can control, such as naming conventions and language, communication between editors and authors, and the pace of the editorial process (Wallace 2020).

Some publishers have created special sections on their publishing platforms focused on manuscripts that have been reviewed and edited, but not yet laid out or otherwise prepared for publishing. This approach can shave weeks off time to publication, as many of the production systems (layout, proofreading, HTML conversion) have been slowed by work-from-home requirements around the world. At least 94 journals, societies, and publishers also committed to making their COVID-19-related research, data, and publications freely available (Groth 2020). Others have worked to accelerate the peer-review process itself, some pushing turnaround times of 24 hours, which is, of course, significantly faster than the typical days or weeks required before the pandemic.

With each part of the publishing process accelerated, something must give. Unfortunately, in some cases, the failures have been in the research itself and the ability of the peer-review process to identify errors before article acceptance. The trade-offs between speed and efficacy have been noted and criticized, sometimes publicly, particularly with two studies related to a controversial treatment involving hydroxychloroquine and azithromycin. In the first case, a study rushed to publication showing the effectiveness of this treatment was widely reported around the world (and even called a “game-changer” by the then-president of the United States). Unfortunately, this study contained many flaws that were not reported, and it was ultimately retracted by the authors. Of this study, Dr. Michael Erdek of the Johns Hopkins Berman Institute of Bioethics and School of Medicine wrote, “Premature publication of definitive recommendations based on inappropriate conclusions grounded in scant, hastily-acquired data serve only at best to confuse and at worst mislead at a time when tensions are high and need for help is great” (Erdek 2020).

There have been other COVID-19 publishing retractions, but work on hydroxychloroquine has been especially politicized. It is noteworthy, therefore, to also mention a second study involving hydroxychloroquine that was also retracted. In this case, the original (widely publicized) study found that hydroxychloroquine was dangerous to COVID-19 patients but was retracted because the authors could “no longer vouch for the veracity of the primary data sources” (McGinley 2020).

With these major retractions occurring even within peer-reviewed journals, it is no wonder that interest in preprints has grown among health sciences publishers. Indeed, some in publishing are beginning to notice that preprints are getting lots of attention and are being noted in policy discussions, mainly because of the speed to publication—and despite fears around data accuracy (Majumder and Mandl 2020). Preprints are now being recognized by researchers as a way to make an instant difference in the fight against COVID-19 (Growth 2020), and some mainstream publishers are rolling out their own preprint servers, such as Wiley’s expanded use of their Authorea platform’s Under Review badge for COVID-19 preprints.

4. A Survey of Health Care Providers

Increased adoption of preprints by the publishing world does not necessarily mean there is an increase in acceptance of preprints by healthcare workers. As a publisher of two nursing journals, I have been part of the ongoing debate about the use of preprint servers in medical publishing. In November 2019, Sigma Theta Tau International released a podcast containing interviews with a nurse researcher, our (Sigma Theta Tau International’s) journal editors, publishing professionals (including myself), and our society’s repository manager. While I have come across some support for preprint servers within nursing (included in a few of the voices on the podcast), anecdotally, I spoke with many nurse researchers in 2019 and earlier who were against their use in health sciences publishing. But that was before COVID-19.

I wanted to see if I could discern any prevailing attitude changes among the consumers of scholarly papers in the health sciences, so I created a brief survey to assess healthcare workers’ current opinions of preprints to try to determine if those opinions had changed since the onset of the COVID-19 pandemic. Respondents to the survey included healthcare practitioners, educators, administrators, consultants, entrepreneurs, editors, and students (some of whom are researchers themselves).

Methods

This cross‐sectional survey was conducted using a web‐based survey tool. Initially a convenience sample was recruited among personally-known healthcare workers and their colleagues, and the survey was soon thereafter promoted on several popular social media and professional networking sites and through word of mouth.

Survey Tool/Questionnaire Design

The survey was hosted on SurveyMonkey, and the URL was made available via direct contact, email, Twitter, LinkedIn, and Facebook. Hashtags for known medical groups, such as #NursesWhoTweet, #MedTwitter, #nurse, #Nursing, and #Healthcareworkers, were used to help target the posts to the right audience.

Given the demands on healthcare workers during the pandemic, this survey was designed to be short and easy-to-complete. While additional detail could be helpful at identifying sub-trends and discerning more granular information about specific occupations and attitudes, I determined my best chance at getting respondents was to value their time and only ask broad questions while avoiding too much detail. 

Ethical Issues

Survey participation was voluntary and self-selected, and all data were collected anonymously. Respondents were not tracked , and the intent and purpose of the survey were explained in text preceding the questions. Respondents were asked to identify their general profession to confirm their employment in the field of healthcare, but verification of responses was not possible.

Analysis

There were 16.8 million healthcare workers in the United States as of May 2018, although the Bureau of Labor Statistics includes administrative and non-practitioner jobs in their industry count (Kaiser 2018). The 118 responses to this survey were enough to provide a 95% confidence level in the results with a 9% margin of error using Survey Monkey’s provided calculator (SurveyMonkey n.d.). Outside the statistical magic making those numbers seem somewhat reasonable, how can we suppose that 118 responses can possibly express the sentiment of almost 17 million people? We cannot, not entirely at least, which is why the recommendations presented in this manuscript are made using what the 118 responses along with my personal experience in publishing (the last 9 years in publishing for nurses). However, the survey responses were consistent enough to tell us what, with some context, moving forward might look like.

5. Survey Questions and Findings

Question 1: What is your opinion regarding the use of preprint servers for medical and nursing research manuscripts?

Two responses were offered for this question:

• I fully support the use of preprint servers to disseminate important medical and nursing research findings as quickly as possible.
• I am against the use of preprint servers to disseminate medical and nursing research due to the potential dangers of posting un-vetted findings.

The results, shown in Figure 1, backed up the anecdotal reports I gathered in the summer and fall of 2019: 65% of respondents chose the second option, “I am against the use of preprint servers.” This question goes directly to the heart of the matter and perhaps squashes any thoughts that preprints are going to become a standard in medical publishing anytime soon. Taken another way, however, 35% of respondents in favor of the use of preprints for medical research might very well be enough to support a healthy preprint ecosystem and provide a solid base for future growth. It seems clear that such growth will not happen right away, but it is worth watching for and, perhaps, repeating a similar survey in the future to see if this grows occurs.

Figure 1: Responses to Question 1.

Question 2: If asked Question 1 in late 2019 (before the emergence of COVID-19), would you have answered in exactly the same way?

Two responses were offered for this question:

• Yes, my answer would have been the same as it is now—my opinion has not changed.
• No, my answer would have been different than it is now—my opinion has changed.

Eighty-six percent of respondents stated their answer would have been the same now as before COVID-19; their opinions have not changed as a result of the pandemic (Figure 2). Frankly, I expected the pandemic would have had a greater effect on healthcare providers, but it appears they are largely sticking to their original opinions (for and against) with very little movement.

Figure 2: Responses to Question 2.

Question 3: To what extent has the COVID-19 pandemic shaped your opinion?

As shown in Figure 3, this question offered a Likert scale of responses ranging from “Strongly in favor of preprints” to “Strongly against preprints,” with five total options. Given the responses to the prior question, I expected most of these responses to be neutral, and in fact a strong plurality—48% of respondents—were not swayed by COVID-19. Twenty percent either strongly or moderately moved against preprints, but 30% moved in favor of preprints. Even if the respondents were not ready to support the use of preprints at the time the survey was administered, it appears there may be some softening of opinion brought on by the pandemic.

Figure 3: Responses to Question 3.

Question 4: What is your role in healthcare? (optional)

This question was made optional because it is not necessarily critical to understanding the opinions of healthcare worker; however, it sheds some light on whether respondents might be consumers of medical research or creators of this research.

Shown in Figure 4, 46% of the respondents were in clinical practice (article consumers), 39% were teachers (likely consumers), 31% were in research (content creators), and the final 22% were “other.” Within “other,” roles were mainly healthcare administrators, consultants, and editors (or other medical publishing professionals), which could also be put in the article consumer category.

Figure 4: Responses to Question 4.

6. Limitations

This article outlines several commonly perceived advantages and disadvantages of preprints, but the survey was not detailed enough to determine which of those factors influenced respondents’ opinions prior to the COVID-19 pandemic. While the survey results indicate that the majority of researchers (64%) were not in favor of preprints before COVID-19 and remain against them, it is unknow what made them dislike preprints originally.

Additionally, from a design perspective, the survey was limited by self-selection. While attempts were made to solicit responses from a wide range of healthcare workers, there were many factors limiting participation. Results may be skewed by lack of participation by certain groups who may have seen the survey but chose not to participate.

Furthermore, the intentionally brief nature of the survey was itself a weakness. The survey was only able to scratch the surface of the respondents’ opinions and was not able to allow particularly detailed analysis.

Most notably, the survey does not distinguish between types of manuscripts posted to preprint servers. Some respondents might indicate their acceptance of pure research preprints (analyzing protein chains on the surface of SARS-CoV-2, for instance) while at the same time being against preprints discussing treatments and patient care. Without being able to distinguish between the two types of papers, respondents were forced to indicate their strongest feelings with no nuance.

These limitations could explain a discrepancy between responses to this survey and the large increases in preprints since the beginning of the pandemic. As discussed earlier, this survey’s respondents who identified themselves as researchers were far less likely to approve of the use of preprints. Given the dramatic increase in the number of preprints submitted since March of 2020 (Fraser and Kramer 2020), however, clearly not all researchers feel that way. Without being able to distinguish among types of researchers (those researchers doing “hard science” analysis compared to those researching treatment options), I can only speculate that my survey responses consisted mainly of those researching treatment options—those less likely to approve of preprints—and contained fewer hard-science researchers who have already made great use of preprints. (This assumption would match the type of researchers in my professional network, many of whom specialize in patient care and treatment, who were likely to see and share the survey invitation.)

7. Conclusions

Interestingly, of those who responded in favor of preprints, 78% of them said their answer would have been the same before the pandemic. However, 66% of those respondents said COVID-19 had still influenced their opinion strongly or moderately in favor of preprints, implying that the disease has made their belief in the value of preprints stronger than pre-pandemic.

Of those who responded against preprints, 91% have not changed their minds since the pandemic, and 58% of them said COVID-19 had no influence on their thinking. They did not like preprints before, and they do not like them now.

Translation: While those opposed to preprints outnumber those in favor 65% to 35%, this pandemic experience is making the “believers” believe even more strongly. Will this lead to increased advocacy for preprints and a push for their acceptance? As mentioned previously, this group could be big enough (35% of healthcare workers as extrapolated from survey results, previous caveats about statistics notwithstanding) to support a strong preprint community and having their opinions strengthened by the pandemic could cause their influence and advocacy to grow.

What could be working against this, however, is the tepid support for preprints among researchers—the creators of content for preprint or traditional publication. 64% of researchers in my survey were against preprints; they have not been swayed by COVID-19 (89% have the same opinion as before the pandemic), and 75% of them were neutral or moved against preprints since the start of the pandemic. As noted, I suspect they may be a particular type of researcher worried about patient care.

It seems there is a disconnect between those who consume research (who comprise the majority of those approving of preprints) and those who create the research (who largely do not like preprints). How will this disconnect between supply and demand play out? My opinion is that researchers will want to go where their work is most likely to be read and disseminated. For now, that is, and will likely remain, with traditional peer-reviewed journals. But if article consumers continue to nurture medical preprints, they may eventually start to win over the researchers as those preprint platforms gain more acceptance.

8. Recommendations for Publishers/Preprint Servers

Although preprints do not seem set to become the norm in scholarly medical publishing, there are clear signs (the growth of preprint servers in the past few years, the number of survey respondents indicating more acceptance due to COVID-19, and the dramatic increase in preprint submissions starting in 2020) indicating their continued presence and acceptance. With that in mind, I have come up with three recommendations that address some of the ethical issues facing publishers.

1. More Clearly Identify Preprints
While many preprint servers include disclaimers noting the lack of peer review, journalists will continue to quote preprints covering trending topics, and not all of them will pass along those preprint disclaimers to their readers. While the owners of preprint servers cannot police outside reporting based on their manuscripts, the existing warnings could very easily be made more prominent. Some are posted using red fonts, which is a start, but many disclaimers are barely distinguishable from the article abstracts. Perhaps applying a “DRAFT” watermark to web pages and PDF versions of preprints might be an easy-to-implement yet highly-visible design change to increase visibility of the preprint designation.

Clearly marking preprints can go a long way in maintaining credibility of the sites that host them as well as the practice of posting preprints in general. As recommended by COPE, “Preprint servers should mark content clearly to indicate it has not undergone peer review” (Tierney 2019, Slide 11). There should be an increased emphasis on the “clearly” part of that statement.

2. Assess the Success and Sustainability of Accelerated Peer Review—and Peer Review Itself
“Many journals have deployed new processes to expedite the review of COVID-19 papers. This situation will present some interesting case studies in how long a journal can sustain an ‘emergency’ process” (Cochran 2020).

News of some journals implementing 24- to 48-hour turnaround time for peer review is impressive. Sustainability is clearly a concern, as many journals have been finding it increasingly difficult to attract and maintain peer reviewers even before the pandemic. Keeping these reviewers working on turnaround time measured in days rather than weeks will likely lead to reviewer fatigue and burnout, exacerbating existing strains on the peer-review system. 

The quality of this expedited peer review should be closely watched as well. As has been noted, several high-profile peer-reviewed articles were retracted early in the COVID-19 pandemic. Even traditional peer review can suffer from lack of rigor and thoroughness unrelated to COVID-19. “[A] lot of peer review reports are skimpy—even when there are glaring problems with the manuscript” (Bastian 2020). If publishers want to maintain a quality distinction between preprints and traditional journals, the value of the product of traditional journals must consistently exceed that of preprints.

3. Be Open to Supporting the Use of Preprints

This may seem like an odd recommendation given the number of survey respondents who came out against preprints, but as previously noted, in a field as large as healthcare, 35% of workers potentially in favor of preprints could be more than enough to support a thriving community of medical preprints. Publishers should continue to watch, and more thoroughly measure, attitudes towards preprints. If the sentiment of researchers and research consumers continues to move in favor of preprints, it would behoove publishers to find ways to support the use of preprints, such as backing or partnering with existing preprint servers, creating or acquiring their own preprint servers, or finding creative ways to integrate the use of preprints into the normal article workflow. There has been some movement in this area (Wiley’s use of Authorea to support COVID-19 papers is one example; linking preprint articles with their final published version is another), and more work can be done to create “official” paths from preprint to publication.  

Allowing for the potential publication of manuscripts that have been submitted to preprint servers instead of automatically disqualifying them is a simple first step. This could include applying more editorial scrutiny to these manuscripts if it helps readers and editorial boards feel confident that problematic manuscripts will never be published. Examples of editorial scrutiny could include requiring additional peer reviewers to weigh in on manuscripts that have been posted to preprint servers. Creating a peer review “A-team” of high-performing reviewers who have received some training around preprints could be another option. These extra steps could be employed to reassure journal editors and readers that manuscripts coming from preprint servers are up to the quality standards of the journal. Such reassurances could go a long way in overcoming some of the negative feelings surrounding preprints, particularly among researchers. It needs to be noted that these options would require significant effort from the publisher and journal staff, which could be daunting given the challenges they are already facing finding enough qualified peer reviewers for their existing manuscripts. These efforts would need to be deliberate and have the full support of the publisher if they are to succeed.

An alternative, called “refereed preprint,” has been used in life-sciences publishing through the publisher-supported website Review Commons (PLOS 2019). This process assigns peer reviewers prior to journal submission, giving journal editors an additional insight into the quality of manuscripts before assigning their own set of reviewers.

A particularly forward-thinking solution was undertaken by the Royal Society back in 2017 with their assignment of a Preprint Editor whose job was to find existing preprint manuscripts and encourage submission to their journal, Proceedings B. (Russell et al. 2021) This “pre-scrutiny” could help legitimize preprints with the introduction of professional editors to the publication process. 

Established publishers have the knowledge, tools, and expertise to solve many of the problems that have been identified with preprints, but if publishers refuse to engage with preprints in any way, there is nothing being done to improve the quality of research within preprints. If publishers underestimate preprints, they may end up ceding this area to upstarts and other companies looking to disrupt the journal space.

9. Author's Comments, February 2023

Since 2020, submissions to preprint servers have continued to grow—along with their acceptance. A comparison of recent nursing manuscript submissions to preprint servers like MedArxiv to two or three years ago clearly shows this growth. Similarly, the acceptance of preprints by most major publishers has also grown. Spring-Nature, Wiley, and Elsevier have bought or started their own preprint servers (Schonfeld and Rieger 2020), and many publishers have policies allowing preprints to be considered for publication in at least some of their journals. A few years ago, this was more controversial and less common.

The survey was conducted, and this article subsequently written, over the summer of 2020 when the world was still very much in the early throes of the pandemic. When I was asked to revise the manuscript for publication in late 2022, I reread the piece and reviewed the survey results to look for any detail that needed to be updated for publication in 2023. While it was not possible to redo the study or host a follow-up survey, continuing to work with nurses and other healthcare professionals over the past two-and-a-half years gives me confidence in its original results. While this confidence is based only on anecdotal evidence, my interactions and discussions with nurses lead me to believe that while many healthcare providers remain cautious or even skeptical regarding preprints, a sizable number of nurses are warming to them.

I similarly have confidence that my original references remain relevant and accurate. Preprints’ primary weaknesses remain largely as outlined above, and my recommendations for how publishers can respond to these weaknesses are unchanged. I believe publishers can improve our existing journal publishing processes while also working to reduce some of preprints’ deficiencies and even making room for them in our publishing programs. As noted above, the largest publishers are already doing just that.

Preprints will likely never replace scholarly journals, but as long as scholarly journals continue to struggle retaining qualified peer reviewers, for instance, there will be a desire for alternative publishing platforms with which researchers can share their work quickly. Publishing accurate information is an ethical imperative, particularly in the health sciences sector of the industry. Preprints have shown great promise along with some troubling weaknesses—weaknesses we as publishing professionals have the ability to eliminate or at least significantly minimize by applying our expertise. Scholarly publishing can only be improved by our efforts in this area.

References

Annas, George. J. 2017. “Health law.” In Encyclopedia Britannica. Last modified May 18, 2017. https://www.britannica.com/science/health-law#ref412617.

Bastian, Hilda. 2016. “Breaking Down Pros and Cons of Preprints in Biomedicine.” Absolutely Maybe (blog), PLOS Blogs. May 1, 2016. https://absolutelymaybe.plos.org/2016/05/01/breaking-down-pros-and-cons-of-preprints-in-biomedicine/.

Bastian, Hilda. 2020. “Reasons to Worry Less About the Explosion of Preprints.” Absolutely Maybe (blog), PLOS Blogs. May 25, 2020. https://absolutelymaybe.plos.org/2020/05/25/reasons-to-worry-less-about-the-explosion-of-preprints/.

Chiarelli, Andrea, Johnson, Rob, Richens, Emma, and Stephen Pinfield. 2019. “Accelerating Scholarly Communication: The Transformative Role of Preprints.” Zenodo. September 24, 2019. https://doi.org/10.5281/zenodo.3357727.

Cochran, Angela. 2020. “What Will We Learn About Scholarly Publishing as a Result of COVID-19?” The Scholarly Kitchen. April 28, 2020. https://scholarlykitchen.sspnet.org/2020/04/28/what-will-we-learn-about-scholarly-publishing-as-a-result-of-covid-19/.

COPE Council. 2018. “COPE Discussion Document: Preprints.” 1st ver. COPE. March 2018. https://doi.org/10.24318/R4WByao2.

COPE Council. 2017. “COPE Ethical Guidelines for Peer Reviewers.” 2nd ver. COPE. September 2017. https://doi.org/10.24318/cope.2019.1.9.

Davey, Melissa. 2020. “Australia's chief scientist warns against claims of breakthroughs on coronavirus cures.” The Guardian. April 29, 2020. https://www.theguardian.com/australia-news/2020/apr/30/australias-chief-scientist-warns-against-claims-of-breakthroughs-on-coronavirus-cures.

The Economist. 2020. “Smokers seem less likely than non-smokers to fall ill with covid-19.” May 2, 2020. https://www.economist.com/science-and-technology/2020/05/02/smokers-seem-less-likely-than-non-smokers-to-fall-ill-with-covid-19.

Erdek, Michael. 2020. “Publishing During the COVID-19 Epidemic: Prompt Conclusions or Quality Studies?” Johns Hopkins Berman Institute of Bioethics and School of Medicine. March 30, 2020. https://bioethics.jhu.edu/news-events/news/publishing-during-the-covid-19-epidemic-prompt-conclusions-or-quality-studies/.

Fraser, Nicholas, and Bianca Kramer. 2020. “Covid19_preprints.” figshare. March 26, 2020. https://doi.org/10.6084/m9.figshare.12033672.v18

Garisto, Daniel. 2019. “Preprints Make Inroads Outside of Physics.” APS News 28, no. 9 (October 2019). https://www.aps.org/publications/apsnews/201909/preprints.cfm.

Groth, Mike. 2020. “COVID-19 and rapid publication: The new normal for scholarly communications?” KnowledgeWorks Global Ltd. March 23, 2020. https://www.cenveopublisherservices.com/blog/2020/3/23/covid-19-and-rapid-publication-the-new-normal-for-scholarly-communications.

Howard, Matthew S., Thompson, Kimberly S., Gennaro, Susan, Melnyk, Bernadette M., Sullivan, Dustin, Wielgus, Jamie, and Sarvenaz Sarabipour. 2019. “Preprints in nursing scholarship: SigmaCast, Episode 1.” November 11, 2019. In SigmaCast. Sigma Repository. Podcast, MP3 audio, 1:12:00. https://sigma.nursingrepository.org/handle/10755/19168.

Johnson, Carolyn Y. 2020. “Moderna's coronavirus vaccine shows encouraging early results.” The Washington Post. May 19, 2020.https://www.washingtonpost.com/health/2020/05/18/coronavirus-vaccine-first-results/.

Johnson, Rob, and Andea Chiarelli. 2019. “The Second Wave of Preprint Servers: How Can Publishers Keep Afloat?” The Scholarly Kitchen. October 16, 2019. https://scholarlykitchen.sspnet.org/2019/10/16/the-second-wave-of-preprint-servers-how-can-publishers-keep-afloat/. 

Kaiser Family Foundation. 2018. “Total Healthcare Employment.” May 2018. https://www.kff.org/other/state-indicator/total-health-care-employment/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D.

Leopold, Seth S., Haddad, Fares S., Sandell, Linda J., and Marc Swiontkowski. 2019. “Editorial: Clinical Orthopaedics and Related Research, The Bone & Joint Journal, The Journal of Orthopaedic Research, and The Journal of Bone and Joint Surgery Will Not Accept Clinical Research Manuscripts Previously Posted to Preprint Servers.” Clinical Orthopaedics and Related Research 477, no. 1 (January 2019): 1-4. https://doi.org/10.1097/CORR.0000000000000565.

Majumder, Maimuna S., and Kenneth D. Mandl. 2020. “Early in the epidemic: Impact of preprints on global discourse about COVID-19 transmissibility.” The Lancet 8, no. 5 (May 2020): E627-E630. https://doi.org/10.1016/S2214-109X(20)30113-3.

McGinley, Laurie. 2020. “Researchers retract study that found big risks in using hydroxychloroquine to treat covid-19.” The Washington Post. June 4, 2020. https://www.washingtonpost.com/health/2020/06/04/researchers-retract-study-that-found-big-risks-using-hydroxychloroquine-treat-covid-19/.

AUPresses Acquisitions Editorial Committee. (2016). Best Practices for Peer Review. 1st ed. New York: Association of University Presses. http://www.aupresses.org/resources/for-members/handbooks-and-toolkits/peer-review-best-practices.

PLOS (The Public Library of Science). 2019. “Review Commons is now live.” The Official PLOS Blog (blog), PLOS Blogs. December 9, 2019. https://theplosblog.plos.org/2019/12/review-commons-is-now-live/.

Pradhan, Prashant, Pandey, Ashutosh K., Mishra, Akhilesh, Gupta, Parul, Tripathi, Praveen K., Menon, Manoj B., Gomes, James, Vivekanandan, Perumal, and Bishwajit Kundu. 2020. “Uncanny similarity of unique inserts in the 2019-nCoV spike protein to HIV-1 gp120 and Gag.” bioRxiv. January 31, 2020. https://www.biorxiv.org/content/10.1101/2020.01.30.927871v2.

Russell, Byron, Sack, John, McGonagle-O'Connell, Alison, and Tony Alves. 2021. “Guest Post - Publishers Integrate Preprints Into Their Workflows.” The Scholarly Kitchen. September 13, 2021. https://scholarlykitchen.sspnet.org/2021/09/13/guest-post-publishers-integrate-preprints-into-their-workflows/.

Schonfeld, Roger C., and Oya Y. Rieger. 2020. “Publishers Invest in Preprints.” The Scholarly Kitchen. May 27, 2020. https://scholarlykitchen.sspnet.org/2020/05/27/publishers-invest-in-preprints/.

Sheldon, Tom. 2018. “Preprints could promote confusion and distortion.” Nature 559, no. 445. Last modified July 25, 2018. https://www.nature.com/articles/d41586-018-05789-4.

Shmerling, Robert H. 2015. “First, do no harm.” Harvard Health Blog (blog), Harvard Health Publishing. June 22, 2020. https://www.health.harvard.edu/blog/first-do-no-harm-201510138421.

SurveyMonkey. n.d. “Margin of error calculator.” https://www.surveymonkey.com/mp/margin-of-error-calculator/. Accessed  May 20,2020.

Tierney, Heather L. 2019. “Preprints and Ethics: Thoughts from the Committee on Publication Ethics (COPE).” Presentation at the World Conference on Research Integrity, Hong Kong, June 3, 2019. https://publicationethics.org/sites/default/files/heather_tierney_preprints_tierney_wcri_2019.pdf.

Wallace, Jasmine. 2020. “Editorially Surviving an Outbreak.” The Scholarly Kitchen. March 18, 2020. https://scholarlykitchen.sspnet.org/2020/03/18/editorially-surviving-an-outbreak/.

Wilson, F. P. 2020. “No, Smoking Won't Save You From COVID-19.” Methods Man Blog (blog), The Methods Man. May 14, 2020. https://www.methodsman.com/blog/covid-19-smoking.

World Medical Association. 2018. “WMA International Code of Medical Ethics.” Last modified July 9, 2018. https://www.wma.net/policies-post/wma-international-code-of-medical-ethics/.