When done well, systematic reviews with meta-analysis provide a comprehensive view of the current state of the evidence on a topic as well as the general strength and direction of findings. Sometimes, however, what they find is that little evidence so far exists, or that the findings are relatively heterogenous. In these cases, the addition of more studies will likely improve their precision; thus, future updates are warranted to reanalyze the data with the infusion of newly published studies and grey literature. (Check out our series on living systematic reviews for one way these reviews could be kept continuously up-to-date – Part 1, Part 2, Part 3).
A recent analysis published in the September issue of the Journal of Clinical Epidemiology supports this concept. Analyzing the original and updated versions of 30 meta-analyses published between 1994 and 2018 across 19 countries, Gao and colleagues found the following:
- The average time from original publication to update was 4 years, 8 months.
- Most of the updates (80%) included more randomized controlled trials than the original. In the majority of cases (73%), an update also led to a higher total number of patients analyzed.
- The proportion of reviews reporting the methodological quality of included studies was slightly higher among the updates (76.7%) than the original publications (70%).
- The quality of most included trials was low, and in most cases, the authors did not report the results of the assessment of individual items within their methodological appraisal.
 |
| A figure from Gao et al. shows the frequencies of publication years of original systematic reviews analyzed (in blue) and their updates (in green). Click to enlarge. |
Among the 30 pairs of included SRs, the authors identified 130 comparable outcomes in just over half (16) of the reviews that could be analyzed for changes over time; most of these (74.6%) were binary outcomes. Overall, just three (2.3%) of the outcomes had a significant change that affected the estimates to a statistically significant degree. In 88.3% of the 94 comparable outcomes that incorporated new evidence, the precision improved (i.e., the width of the confidence interval narrowed). Precision was reduced in five outcomes (5.3%) and did not change in the remaining six (6.4%).
The authors conclude that while update of meta-analyses within systematic reviews often improves the precision of an estimate (as assessed by the width of the confidence interval around it), it rarely changes the estimate itself. In addition, systematic review authors should take care to more thoroughly assess and report the methodological appraisal of individual items within each included study. This may serve as a guide for determining whether a future update may dramatically impact current findings, and how frequently updates should be conducted.
Gao, Y., Yang, K., Cai, Y., Shi, S., Liu, M., Zhang, J... & Song, F. Updating systematic reviews can improve the precision of outcomes: A comparative study. J Clin Epidemiol, 2020; 125: 108-119.
Manuscript available here.
