The evidence that underpins modern medicine is only as strong as the individual studies fed into it, and a growing body of research shows that a meaningful share of clinical trials carry problems reviewers have struggled to detect. A new tool aims to close that gap before flawed results can shape treatment guidelines and, ultimately, patient care.
Called INSPECT-SR, the instrument was developed by researchers at the University of Manchester, Cochrane and the University of Colorado, working with more than 40 institutions around the world. It gives systematic reviewers a structured, transparent way to judge whether a randomized controlled trial is trustworthy, and its developers describe it in a paper published in BMJ.
Why a single flawed trial can travel far
Systematic reviews, and the meta-analyses built on them, sit near the top of the evidence hierarchy. Guideline panels, regulators and clinicians lean on them to decide which treatments work. That authority depends entirely on the studies they gather. Problematic trials, whether undermined by human error or by research misconduct, can skew pooled estimates and push recommendations toward conclusions that are inaccurate or even harmful.
What makes the problem especially difficult is how long it can stay hidden. Issues with a trial may go unrecognized for years, and many problematic studies are never formally retracted. Because retraction notices form an incomplete record, the true scale of the issue is hard to pin down. Reviewers, meanwhile, have historically had only limited and inconsistent methods for evaluating whether a trial's data can be relied upon at all.
Ivermectin and COVID-19: a cautionary example
The consequences are not hypothetical. A recent case involved systematic reviews of ivermectin as a treatment for COVID-19. According to health authorities in the United States, the United Kingdom and the European Union, some of the trials included in those reviews were not authentic. When subsequent high-quality trials were run, they suggested little or no benefit from the drug. The episode illustrated how unreliable primary studies can propagate through reviews, public messaging and clinical practice long before the underlying data are called into question.
Inside the tool: 21 checks across four areas
INSPECT-SR addresses that vulnerability with a deliberately systematic process. It walks reviewers through 21 structured checks, organized into four domains:
- Post-publication notices — whether a trial has been flagged, corrected or otherwise annotated after it appeared in the literature.
- Study conduct, governance and transparency — how the trial was run and documented.
- Text and figures — the published presentation of the research itself.
- Study results — the numbers and outcomes the trial reports.
By standardizing those checks, the framework is intended to remove some of the guesswork from a task that reviewers previously approached in ad hoc ways. The goal is consistency: two review teams examining the same trial should arrive at comparable conclusions about whether its evidence belongs in a synthesis.
Trustworthiness is not the same as bias risk
The distinction matters. Existing instruments tend to focus on risk of bias or methodological quality, asking whether a study was designed and reported well enough to support its conclusions. INSPECT-SR instead homes in on whether the trial data themselves can be trusted.
Its authors are careful about what the tool does and does not claim. The framework does not presume misconduct or imply deliberate wrongdoing by researchers; it is built to surface problematic trials of many kinds. At the same time, it can help identify fraudulent studies alongside trials compromised by error, giving reviewers a defensible basis for excluding suspect evidence or testing how their findings hold up without it.
Built with and for the review community
The tool's credibility rests partly on who shaped it. More than 150 integrity and health research experts from around the world contributed to its development, according to the team. Dr. Jack Wilkinson, the lead researcher, said researchers can use INSPECT-SR to assess the trustworthiness of randomized controlled trials and to identify problematic studies, knowing that it carries the backing of the systematic review community.
The framework was designed to be rigorous, systematic and transparent, qualities the authors argue are essential if trustworthiness assessments are to hold steady from one review team to the next rather than depending on individual judgment.
From tool to standard practice
The research team is encouraging reviewers, guideline developers and publishers to adopt INSPECT-SR, and expects it to become the standard for assessing the trustworthiness of randomized controlled trials. Lisa Bero, a fellow author on the paper, framed the stakes succinctly: systematic reviews are only as credible as the studies they include.
Wider uptake would mean questions about a trial's authenticity get asked at the point of synthesis, a shift from the current pattern in which doubts often surface only after a review has already influenced guidance.
What it means for patients and guidelines
For clinicians and patients, the value lies in what gets filtered out. Guidance built on a contaminated evidence base can steer people toward treatments that do not work, or away from ones that do. A standardized screening step cannot clean up unreliable research on its own, but it gives the people who synthesize evidence a clearer, more repeatable way to decide which trials deserve a place in the analysis and which should be set aside until their data can be verified.
As the ivermectin experience showed, the cost of missing those signals is measured in wasted time, squandered resources and eroded public confidence. The developers of INSPECT-SR are betting that a checklist built by the review community, for the review community, can catch more of them earlier.
This article is based on reporting by Medical Xpress. Read the original article.
Originally published on medicalxpress.com
