WHO Guidelines are Considering Health Equity More Frequently, but Reporting of Judgments is Often Incomplete

The GRADE evidence-to-decision (EtD) framework was developed as a way to more explicitly and transparently inform the considerations of the implications of clinical recommendations, such as the potential positive or negative impacts on health equity. A new analysis of World Health Organization (WHO) guidelines published between 2014 and 2019 - over half (54%) of which used the EtD framework - examines the consideration of health equities in the guidelines' resulting recommendations.

Dewidar and colleagues found that the guidelines utilizing the EtD framework were more likely to be addressing health issues in socially disadvantaged populations (42% of those developed with the EtD versus 24% of those without). What's more, the use of the EtD framework has risen over time, from 10% of guidelines published in 2016 (the year of the EtD's introduction) to 100% of those published within the first four months of 2019. Use of the term "health equity" increased to a similar degree over this period.

Just over one-third (38%) of recommendations were judged to increase or probably increase health equity, while 15% selected the judgment "Don't know/uncertain" and 8% provided no judgment. Just over one-quarter (28%) of the recommendations utilizing the EtD framework provided evidence for the judgment. When detailed judgments were provided, they were more likely to discuss the potential impacts of place of residence and socioeconomic status and less likely to explicitly consider gender, education, race, social capital, occupation, or religion.

Click to enlarge.

The authors conclude that while consideration of the potential impacts of recommendations on health equity has increased considerably in recent years, reporting of these judgments is still often incomplete. Reporting which published research evidence or additional considerations were used to make a judgment, as well as considering the various PROGRESS factors (Place, Race, Occupation, Gender, Religion, Education, Socioeconomic status, and Social capital) will likely improve the transparency of recommendations in future guidelines where health equity impacts are of concern.

Dwidr, O., Tsang, P., León-Garcia, M., Mathew, C., Antequera, A., Baldeh, T., ... & Welch, V. 2020. Over half of WHO guidelines published from 2014 to 2019 explicitly considered health equity issues: A cross-sectional suvey. J Clin Epidemiol 127:125-133.

Manuscript available from the publisher's website here.

Timing and Nature of Financial Conflicts of Interest Often Go Unreported, Systematic Survey Finds

The proper disclosure and management of financial Conflicts of Interest (FCOI) within the context of a published randomized controlled trial is vital to alerting the reader to the sources of funding for the research and other financial factors that may influence the design, conduct, or reporting of the trial.

A recently published cross-sectional survey by Hakoum and colleagues examined the nature of FCOI reporting in a sample of 108 published trials found that 99% of these reported individual author disclosures, while only 6% reported potential sources of FCOI at the institutional level. Individual authors reported a median of 2 FCOIs. Among the 2,972 FCOIs reported by 806 individuals, the greatest proportion came from personal fees other than employment income (50%) and from grants (34%). Further, of those disclosing individual FCOI, a large majority (85%) were provided by private-for-profit entities. Notably, only one-third (33%) of these disclosures included the timing of the funding in relation to the trial, 17% reported the relationship between the funding source and the trial, and just 1% reported the monetary value.

Click to enlarge.

 

Using a multivariate regression, the authors found that the reporting of FCOI by individual authors was positively associated with nine factors, most strongly with the authors being from an academic institution (OR: 2.981; 95% CI: 2.415 – 3.680), with the funding coming from an entity other than private-for-profit (OR: 2.809; 95% CI: 2.274 – 3.470), and the first author’s affiliation being from a low- or middle-income country (OR: 2.215; 95% CI: 1.512 – 3.246).

 

More explicit and complete reporting of FCOIs, the authors conclude, may improve readers’ level of trust in the results of a published trial and in the authors presenting them. To improve the nature and transparency of FCOI reporting, researchers may consider disclosing details related to the funding’s source, including the timing of the funding in relation to the conduct and publication of the trial, the relationship between the funding source and the trial, and the monetary value of the support.

Hakoum, M.B., Noureldine, H., Habib, J.R., Abou-Jaoude, E.A., Raslan, R., Jouni, H., ... & Akl, E.A. (2020). Authors of clinical trials seldom reported details when declaring their individual and institutional financial conflicts of interest: A cross-sectional survey. J Clin Epidemiol 127:49-58.

Manuscript available from the publisher's website here. 

Assessing Health-Related Quality of Life Improvement in the Modern Anticancer Therapy Era

Recent breakthroughs in anticancer therapies such as small-molecule drugs and immunotherapies have made improvements in Health-Related Quality of Life (HRQOL) possible among cancer patients over the course of treatment. In a recent paper published in the Journal of Clinical Epidemiology, Cottone and colleagues are the first to propose the framework for assessing the change in HRQOL over time in these patients: Time to HRQOL Improvement (TTI), and Time to Sustained HRQOL Improvement (TTSI).

In the proposed framework, TTI is based on the time to the “first clinically meaningful improvement occurring in a given scale or in at least one among different scales” – for instance, a minimal important difference (MID) of 5 points on the European Organization for Research and Treatment of Cancer (EORTC) Quality of Life Questionnaire – Core 30 (QLQ-C30). The authors suggest utilizing the first posttreatment score as the baseline measurement for monitoring improvements over time. “Sustained improvement” was defined as the first improvement that is not followed by a deterioration that meets or exceeds the MID.

 

The use of Kaplan-Meier curves and Cox proportional hazards is inappropriate for these outcomes, the authors argue, as it does not allow for possible competing events, such as disease progression, toxicity, or the possibility of an earlier improvement in another scale when multiple scales are used. They propose the use of the Fine-Gray model for the evaluation of TTI and TTSI and pilot it with a case study of 124 newly diagnosed chronic myeloid leukemia patients undergoing first-line treatment with nilotinib.

Time To Improvement (TTI) and Time to Sustained Improvement (TTSI) can be used to elucidate differences in HRQOL responses to treatment based on baseline characteristics. Here, the figure shows TTSI in fatigue scores based on hemoglobin level at baseline. Click to enlarge.

Using this model, the authors found that improvements in fatigue scores appeared more quickly than those in physical functioning when measuring scores from baseline (pre-treatment), but upon using first post-treatment score as the baseline, the differences between improvement rates in fatigue and physical functioning diminished. Additionally, a lower baseline hemoglobin level was associated with earlier sustained improvements in fatigue.

 

While the proposed method of evaluating TTI and TTSI has some limitations, such as lower statistical power than other ways of tracking changes in HRQOL over time, it also has notable strengths. In particular, this method can be used to elucidate differences between treatment approaches that show similar survival outcomes so that the approach with shorter TTI and TTSI can be favored.

Cottone, F., Collins, G.S., Anota, A., Sommer, K., Giesinger, J.M., Kieffer, J.M., ... & Efficace, F. (2020). Time to health-related quality of life improvement analysis was developed to enhance evaluation of modern anticancer therapies. J Clin Epidemiol 127:9-18.

Manuscript available from publisher's website here. 

A New Tool for Assessing the Credibility of Effect Modification Cometh: Introducing the ICEMAN

Effect modification goes by many other names: “subgroup effect,” “statistical interaction,” and “moderation,” to name a few. Regardless of what it’s called, the existence of effect modification in the context of an individual study means that the effect of an intervention varies between individuals based on an attribute such as age, sex, or severity of underlying disease. Similarly, a systematic review may aim to identify effect modification between individual studies based on their setting, year of publication, or methodological differences (often called a “subgroup analysis”).

As many as one-quarter of randomized controlled trials (RCTs) and meta-analyses examine their findings for potential evidence of effect modification, according to a paper by Schandelmaier and colleagues published in the latest edition of CMAJ. However, it is not uncommon for claims of effect modification to be later proved spurious, which may negatively affect the quality of care in those subgroups of patients. Potential sources of these claims range from simple random chance to issues with selective reporting and misguided application of statistical analyses.

Click to enlarge.

In “Development of the Instrument to assess the Credibility of Effect Modification in Analyses (ICEMAN) in randomized controlled trials and meta-analyses,” the authors present a novel tool for evaluating the presence of a potential modifier. While several sets of criteria have been developed in the past for this purpose, the ICEMAN is the first to be based on a rigorous development process and refined with formal user testing.

 

First, the authors conducted a systematic survey of the literature to ensure a comprehensive understanding of the previously proposed criteria for evaluating effect modification. Thirty sets were identified, none of which adequately reflected the authors’ conceptual framework. Second, an expert panel of 15 members was identified randomly from a list of 40 identified through the systematic survey. These experts then pared down the initial list of 36 candidate criteria to 20 required and eight optional items. After developing a manual for its use, the authors tested the instrument among a diverse group of 17 potential users, including authors of Cochrane reviews and RCTs and journal editors using a semi-structured interview technique.

Schandelmaier, S., Briel, M., Varadhan, R., Schmid, C.H., Devasenapathy, N., Hayward, R.A., Gagnier, J., ... & Guyatt, G.H. 2020. Development of the Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN) in randomized controlled trials and meta-analyses. CMAJ 192:E901-906.

Manuscript available at the publisher's website here. 

Rapid, Up-to-Date Evidence Synthesis in the Time of COVID

In emergent situations with sparse and rapidly evolving bodies of research, evidence synthesis programs must be able to adapt to a shortened timeline to provide clinicians with the best available evidence for decision-making. (See our previous posts on rapid systematic review and guideline development, here, here, here, and here). But perhaps no health crisis in the modern era has made this more clear than the coronavirus disease 2019 (COVID-19) pandemic.

Recently, Murad and colleagues published a framework detailing a four-pillar program through which they have been able to synthesize evidence related to the COVID-19 pandemic. This system has been tried and tested within the Mayo Clinic, a multi-state academic center with more than 1.2 million patients per year.

 

Launched within two weeks of the World Health Organization’s declaration of COVID-19 as a pandemic, Mayo Clinic’s evidence synthesis program consisted of four major components:

• What is New?: an automatically generated list of COVID-19-related studies published within the last three days and categorized into topic areas such as diagnosis or prevention
• Repository of Studies: a running list of previously published studies since the first case report of COVID-19, including those that move from the “What is New?” list after three days’ time
• Rapid Reviews: reviews published within three to four days in response to pressing clinical questions from those on the frontlines and utilizing the study repository. To facilitate evidence synthesis, studies are often screened and selected by a single reviewer and evidence is rarely meta-analyzed.
• Repository of Reviews: a collection of reviews including those developed at Mayo and elsewhere, identified in twice-weekly searches and through a list of predetermined websites. To supplement knowledge, some reviews included indirect evidence borrowed from studies of other coronaviruses or respiratory infections, when appropriate.

Click to enlarge.

Within one month of the framework’s establishment, the team had conducted seven in-house rapid reviews and had indexed more than 100 newly published reviews into a database housing over 2,000 total.

 

The authors conclude that while an intensive system such as this may not be feasible in smaller health systems, cross-collaboration and sharing of knowledge can allow for informed and up-to-date clinical care that adapts in the face of a rapidly changing landscape of evidence.

Murad, M.H., Nayfeh, T., Suarez, M.U., Seisa, M.O., Abd-Rabu, R., Farah, M.H.E..., & Saadi, S.M. 2020. A framework for evidence synthesis programs to respond to a pandemic. Mayo Clin Proc 95(7):1426-1429.

Manuscript available at the publisher's website here.

New Elaboration of CONSORT Items Aims to Improve the Reporting of Deprescribing Trials

Deprescribing is the act of withdrawing a treatment prescription from patients for whom a medication has become inappropriate or in whom the risks may now outweigh the benefits. However, trials examining the effects of deprescribing are often complex and multi-faceted, and reporting of these trials can miss important aspects such as patient selection and length of follow-up. 

A recently published paper by Blom et al. used a multistep process to develop a reporting guideline for deprescribing trials based on a systematic review of this body of research, paying close attention to those aspects that most commonly went unreported. The result was an elaboration of the Consolidated Standards of Reporting Trials (CONSORT) statement, with the addition of items reviewed by a panel of 14 experts in the areas of ranging from pharmacology and geriatric medicine to statistics and reporting guidelines. The process, which ended with a one-day face-to-face meeting to approve the elaborated items, also took into account the Template for Intervention Description and Replication (TIDieR) checklist to ensure that a comprehensive list was created.

Click to enlarge.

The panel determined that all items of the original CONSORT checklist are applicable to deprescribing trials, but that certain items required further detail. The CONSORT items that required the most attention with regards to deprescribing studies included the following:

• description of trial design
• participant selection 
• detailed information that would allow replication of the intervention studied
• pre-specification of primary and secondary outcome
• discussion of adverse events and harms, including those related to drug withdrawal
• defined periods of recruitment and follow-up

 

In addition to improving the quality of reporting in deprescribing trials, the authors also recommend increasing the amount of dedicated funds available for deprescribing studies, which are currently scarce and not incentivized by common streams of research funding.

Blom, J.W., Muth, C., Glasziou, P., McCormarck, J.P., Perera, R., Poortvliet, R.K.E..., & Knottnerus, J.A. 2020. Describing deprescribing trials better: An elaboration of the CONSORT statement. J Clin Epidemiol 127: 87-95.

Manuscript available from the publisher's website here.

New Systematic Review Suggests Noncordance with COI Disclosure to Reporting Databases is Widespread, but Methodological Quality of Studies is Variable

Disclosure of conflict of interest (COI) is a major point of concern in the development of guidelines as well as original research papers. Over the years, multiple studies have aimed to elucidate just how closely the disclosures of individual authors tracks with their reported COI in open databases. A new systematic review of 27 such studies, recently published online in the Journal of Clinical Epidemiology, compiles the findings of these studies into some eyebrow-raising statistics while also taking a look at the methodological quality of these studies.

 

In their review, El-Rayass and colleagues found that although the methodological quality for assessing the concordance of authors’ COI disclosures within papers and according to public databases varied widely, a median of 81.2% of authors across 20 studies had “noncorcordant” disclosures, (ranging from 41.8% to 98.6% across all studies) and that more than half (43.4% of all authors) of these were “completely nonconcordant” (ranging from 15% to 89.5% across all studies). What’s more, among seven studies that analyzed company reporting on the individual level, between 23.1% and 85.4% of companies did not report their payments to authors.

Click to enlarge.

For the five studies that analyzed disclosures on the study rather than the individual author level, all found at least some degree of discordance between in-study disclosures and database reports. The rate of nonconcordant disclosures among these studies ranged from 6 to 92.6%

 

The authors note that ulterior motives of authors are just one potential explanation for the high observed rate of nonconcordant COI disclosure and reporting. Vague instructions and parameters set by journals during the article submission process may undermine efforts to transparently report any and all potential sources of conflict, be they financial, intellectual or otherwise. In addition, the authors found that studies of COI reporting that tended to have higher methodological quality also tended to report lower estimates of nonconcordance, meaning that the overall combined estimates may be artificially inflated – for instance, due to some studies not making a distinction about the relevancy of potential COI sources to the topic of the articles analyzed. The authors note potential sources of nondirectional error as well, such as how differences in COI categories between in-paper disclosures and reference databases were handled, which additionally lowers confidence in the current estimate.

Click to enlarge.

In sum, the recent review by El-Rayess et al. points out that issues with concordance between authors’ COI disclosures in their published works seem to be at odds with publicly available reports of these relationships; however, the degree of nonconcordance overall is still uncertain. Those looking to complete future analyses of COI disclosure policies may want to use this paper as a roadmap to improving our certainty in the actual magnitude of the issue.

El-Rayess, H., Khamis, A.M., Haddad, S., Ghaddara, H.A., Hakoum, M., Ichkhanian, Y., Bejjani, M., and Akl, E.A. Assessing concordance of financial conflicts of interest disclosures with payments' databases: A systematic survey of the health literature. J Clin Epidemiol 127:19-28.

Manuscript available at the publisher's website here.