Just as with individual research trials, the replication of a systematic review can shed new light on an existing topic or help further solidify our assessment of the certainty of a body of evidence. However, duplication of efforts that is done unintentionally or without deliberate consideration of methodology (e.g., how similar or different the new review will be in terms of evidence searching, inclusion, and synthesis) is wasteful. How is one to know when the replication of a systematic review is appropriate and warranted?
A new consensus checklist recently published by Tugwell and colleagues in BMJ provides guidance on when - and when not - to conduct a systematic review replication. Driven by a six-person executive team, the checklist was informed by the input of methodologists, including experts in fields ranging from clinical epidemiology to guideline development and health economics, to knowledge users - those involved in the funding, commissioning, and development of systematic reviews. Two patients were involved in the development team and an additional 17 patient and public representatives were consulted for input via survey.
The process culminated in the drafting of the checklist in a face-to-face setting, with an original 12 proposed items solidified into a final four. The items ask whether replication of the systematic review is of high priority (e.g., whether replication results will be expected to guide policymakers or be of relevance to stakeholders), whether there are certain methodological concerns (such as search design, scope of PICOs, etc.) that will be clarified or improved with a replication; whether the implementation of the replication's findings would be expected to have a sizable positive or negative impact on the population or individual level; and whether resources (e.g., time, money) spent on replication would not be better spent on conducting a new review to answer a novel question.
 |
| Click to enlarge. |
The ultimate decision of whether or not to replicate should be informed by the answers to these questions, the authors note, and left up to contextualized judgment rather than a quantitative threshold. Further, some of the items may be of higher or lower relevancy depending on the stakeholders for a specific review topic, and "middle-ground" solutions, such as repeating only the parts of a systematic review in need of replication, should be considered individually. The authors plan to test the usability, acceptability, and usefulness of this newly proposed tool with relevant end-users.
Tugwell, P., Welch, V.A., Karunananthan, S., Maxwell, L.J., Akl, E.A., Avey, M.T., ... & White, H. 2020. When to replicate systematic reviews of interventions: Consensus checklist. BMJ 370:m2864.
Manuscript available from the publisher's website here.
