Abstract
Objective
Evidence of larger drug effects in highly standardized studies (efficacy) compared to clinical routine (effectiveness) is discussed as efficacy‐effectiveness gap. This study aimed to quantify effect size differences of RCTs and non‐RCTs in the treatment of depression with venlafaxine and duloxetine and to identify effect modifying predictors.
Methods
A comprehensive systematic review and meta‐analysis was conducted, including all prospective trials, which evaluated the treatment effects of duloxetine or venlafaxine in patients with depression. The primary outcome was the pre‐post effect size after acute therapy, which were compared between RCTs and non‐RCTs. Moreover, an exploratory analysis of predictors in a mixed meta‐regression model within an information‐theoretic approach was performed.
Results
171 RCTs and 74 non‐RCTs were included. The pre‐post effect size differed significantly between RCTs and non‐RCTs (−3.04 vs. −2.62, Δ = 0.41, p = 0.012, high heterogeneity). Study characteristics were very similar between RCTs and non‐RCTs. Most important variables to predict effect sizes were ‘depression severity’, ‘dose’ and ‘number of participants’.
Conclusion
Despite differences in effect sizes between RCTs and non‐RCTs, study design is not clearly an important predictor for the effect sizes. Our results question the common assumption that non‐RCTs are generally better suited to describe a drug’s effectiveness in clinical practice than RCTs. Future studies and their reporting should put more emphasis on the description of external validity, in order to allow better assessments of clinical relevance.