In a current examine revealed in JAMA Community Open, researchers in contrast and decided the concordance between the examine design, outcomes, and inferences of medical and observational research and meta-analyses that have been first posted as medRxiv preprints and have been later peer-reviewed and revealed in journals.
Background
Preprints posted on servers resembling medRxiv could be a double-edged sword. Whereas entry to non-peer-reviewed medical analysis findings can lead to the speedy dissemination of data and permit the analysis of the examine by the scientific group at massive, open entry to such outcomes with out peer evaluation may additionally propagate misinterpretations, inaccurate outcomes, and defective analysis strategies that may have penalties on well being practices.
The few research which have in contrast the examine design, outcomes, and the foremost inferences throughout preprints and their subsequently revealed variations have largely targeted on these preprints revealed in peer-reviewed journals with excessive influence elements (higher than 10). It may be inferred that the preprints that do finally get revealed in journals with excessive influence elements are already of top of the range and don’t require main alterations after peer evaluation.
To find out the reliability of medical examine outcomes posted in preprints, it is very important examine preprints and their subsequent revealed variations in peer-reviewed journals unbiased of the influence issue.
Concerning the examine
Within the current examine, the researchers used the medRxiv preprint server’s software programming interface to acquire all manuscripts first submitted to the server in September 2020. For the preprints up to date with a more moderen model after September 2020, the latest variations have been obtained.
The examine design of every preprint was manually characterised into one in every of many classes, together with meta-analysis with or and not using a systematic evaluation, medical, observational, modeling research, or different. The preprints have been then matched with their corresponding peer-reviewed journal publications.
To make sure that the modifications made within the up to date variations of the preprint weren’t a results of the peer-review course of, the up to date variations posted after the journal acceptance date have been excluded. The time between the date of posting in medRxiv and the digital publication date was additionally decided. The influence issue of the journals was obtained from the InCites Journal Quotation Studies.
For every preprint-journal article pair, data on the pattern dimension, main measured outcomes, outcomes for every end result, and inferences drawn from the outcomes was in contrast. The pattern dimension for medical and observational research was outlined because the variety of individuals within the cohorts or database, whereas for meta-analyses, the pattern sizes have been based mostly on the variety of research included within the evaluation. The first endpoints included ascertainment time and measurement scale, which included elements resembling mortality charges, odd ratio, and so forth.
Concordance in pattern dimension was decided based mostly on numerical similarity, whereas concordance in main outcomes or endpoints was concluded if the publication didn’t report any extra main outcomes. Discordances in pattern dimension and first endpoints have been investigated additional to find out the sort and causes for the discordance. The interpretations have been concluded to be concordant if comparable statements have been made concerning the leads to the preprint and the publication.
Outcomes
The outcomes reported that as of September 2022, 1077 of the 1399 preprints posted to medRxiv throughout September 2020 had been revealed in peer-reviewed journals. Of those, 547 described meta-analyses, medical trials, or observational research, and 53.6% (293) of those have been associated to coronavirus illness 2019 (COVID-19). The preprint-journal article pairs that reported pattern sizes have been 535, of which 86.4% (462) have been concordant, and 43 out of the 73 discordant pairs reported a bigger pattern dimension within the publication.
Out of the 547 pairs, 97.6% (534) have been concordant of their reviews of the first endpoints, and solely 2.4% have been discordant. For pairs with numerical outcomes, 81.1% (434 out of 535) have been concordant, whereas 101 had discordant outcomes for the first outcomes. Of those 101, the impact estimates of 65.3% (66) have been statistically comparable. The interpretations of the outcomes have been concordant for 96.2% (526) of the 547 pairs.
Conclusions
General, the outcomes advised that regardless of the absence of peer evaluation in preprints posted to servers resembling medRxiv, the examine design, measured main outcomes, outcomes, and inferences drawn from the outcomes have been constant between preprints and their subsequently peer-reviewed and revealed variations.
Greater than three-quarters of the preprint and journal article pairs have been concordant in traits resembling main outcomes, outcomes, and interpretations. Moreover, the discordant pairs within the outcomes have been nonetheless comparable of their interpretations.
