Factors influencing the trustworthiness of non-randomized studies of interventions: a scoping review

Background: Risk of bias assessment tools like ROBINS-I have been developed to identify biases in non-randomized studies of interventions. However, additional factors related to study design, reporting, external validity, statistical precision and other factors not accounted for in risk of bias assessment tools could affect the trustworthiness of non-randomized studies of interventions.Objectives: To identify factors, beyond risk of bias (covered by existing tools e.g., ROBINS-I), that could influence the trustworthiness of non-randomized studies of interventions. Design: Scoping reviewData sources and searches: We searched PubMed and Google Scholar for publications that explicitly discussed trustworthiness of non-randomized studies of interventions, published between 2014 and April 2024.Study selection: A single reviewer screened the titles and abstracts, and the full texts, against the eligibility criteria and a second reviewer conducted quality control for 10% of titles and abstracts and of full-texts.Data extraction: A single reviewer extracted data on the study characteristics and the factors that can influence trustworthiness. All extractions were verified by a second reviewer.Data synthesis: We employed deductive and inductive thematic analysis using NVIVO 14.Results: After screening 6471 titles and abstracts, we identified 48 eligible reports. We identified 35 factors influencing the trustworthiness of non-randomized studies of interventions, grouped into 16 domains and six themes. The themes were: 1) Open Science, 2) Research Question, 3) Study Methodology, 4) Data Source, 5) Findings and Interpretation, and 6) Oversight. Conclusion: There are many factors beyond the traditional bias domains that can guide efforts to assess and improve non-randomized studies of interventions.