Fall 2019 - Scholarship recipients

Contributed by Madelin Siedler, 2019/2020 U.S. GRADE Network Research Fellow

The Eleventh GRADE Guideline Development Workshop was held in Orlando, Florida, this past September. This workshop welcomed 52 participants, including several international attendees from Canada and Brazil. Among these participants were three recipients of a scholarship provided by the Evidence Foundation, which covers the cost of registration. Coming from diverse backgrounds ranging from organizational work to evidence synthesis to policymaking, scholars Faduma Gure, Eric Linskens, and Christian Kershaw presented their proposals for new innovations or opportunities for improving the application and implementation of evidence-based medicine.

Faduma Gure, MSc, a knowledge translation and research specialist for the Association of Ontario Midwives, discussed the challenges of incorporating client perspectives to midwifery guidelines for the organization, which utilizes the GRADE approach. Pregnancy through the post-partum period is often a tumultuous time filled with decision-making, Gure explained, and more can be done to better understand the values and preferences of midwifery clients and to employ an equity lens when formulating recommendations. Gure proposed a solution that includes the development of an equity advisory group consisting of key stakeholders representative of Ontario’s population of midwifery clients. The organization could then involve these stakeholders through the entire guideline development process - from the initial setting of research priorities to the ultimate formulation of recommendations - and elicit important feedback about patient values and preferences as well as the potential impacts of a guideline on various communities.

Eric Linskens, BSc, serves the Minneapolis Veterans Affairs Evidence Synthesis Program and the Minnesota Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center. Linskens presented his current work applying the GRADE approach to existing systematic reviews which did not originally use GRADE. Linskens discussed some of the challenges that his team has faced as part of the initiative, as well as innovative solutions to these issues. For instance, a systematic review conducted by AHRQ may automatically rate down for inconsistency due simply to the inclusion of a sole study, whereas in the GRADE approach, this would not be the case. Additionally, an existing review may break down one clinical question into multiple smaller analyses of comparators or sub-populations, whereas it would be more clinically relevant to use these data to create one larger recommendation. To best solve these issues, Linskens noted, it is important to consider the end-user of any given review or guideline so that their needs can be best met. Additionally, transparently reporting all judgments around the analyses is key. Regarding his time at the workshop, Linskens said, “[i]t was very helpful to work through examples with the GRADE workshop facilitators in small group sessions. They answered our questions as they came up.”

Christian Kershaw, PhD, is a molecular neuroscientist who now works as a health policy analyst for CGS, a Medicare fee-for-service contractor. Dr. Kershaw used her personal experience transitioning from bench science to policymaking to inspire her presentation on the utility of cross-functional teams in medicine and healthcare policy. To develop a cross-functional team, Dr. Kershaw explained, it is best to identify a problem that would best be solved by a group of individuals with heterogeneous skills and backgrounds that would each uniquely serve a common goal or purpose. As an example, Kershaw discussed the development of a team to standardize the way information is used to form coverage decisions as part of the 21st Century Cures Act. The team is comprised of a medical doctor to understand the need for and content of the policies; an outreach and education specialist to understand their legal implications; and a basic research scientist to compile and assess the information. Leveraging individual team members’ strengths and encouraging innovation are keys to success when working in a cross-functional team. “I was impressed with the versatility of the GRADE framework,” Kershaw noted. “It was very informative to learn all of the different ways that the conference attendees were using GRADE to suit their projects.”

If interested in applying for a scholarship to future GRADE workshops, more details can be found here: https://evidencefoundation.org/scholarships.html. Please note the deadline for applications to our next workshop in Phoenix, AZ will be December 4, 2019.

Research Shorts: When continuous outcomes are measured using different scales

Contributed by M. Hassan Murad, MD

Outcomes of great importance to patients, such as quality of life and severity of anxiety or depression, are often measured using different scales. When an outcome is measured using several scales across trials, it requires standardization to be pooled in a meta-analysis.

Common methods of standardization include using the standardized mean difference (SMD), converting continuous data to binary relative and absolute association measures, the minimally important difference (MID), the ratio of means, and transforming standardized effects back to original scales. The underlying assumption in all these methods is that the different scales measure the same construct. This paper, in BMJ, describes these methods and suggests approaches for interpretation.


Reference: Murad Mohammad Hassan, Wang Zhen, Chu Haitao, Lin Lifeng. When continuous outcomes are measured using different scales: guide for meta-analysis and interpretation BMJ 2019; 364 :k4817. https://www.bmj.com/content/364/bmj.k4817 

Research Shorts: Surrogate endpoints using the example of hepatitis C virus

Contributed by Claudia Dobler, MD, PhD

2018 U.S. GRADE Workshop Scholarship Recipient

Surrogate endpoints (for example laboratory or imaging results) are commonly used in clinical trials, as they require less participants and can be done in a shorter period of time compared to trials that use clinically important outcome measures such as mortality. When evidence for an intervention is almost exclusively based on trials that used surrogate outcomes, it is challenging for decision makers to determine the appropriateness of the use and reimbursement of an intervention. The common tendency in evidence-based medicine is to view results based on surrogate endpoints as less certain than results based on long term, final patient-important outcomes. The authors of this paper use the contemporary and highly debated example of the surrogate endpoint ‘sustained viral response’ (i.e., viral eradication considered to represent successful treatment) in patients treated for chronic hepatitis C virus infection to demonstrate how the validity of a surrogate endpoint can be critically appraised to assess the trustworthiness of the evidence and the implications for decision-making. They outline how the GRADE system for determining the certainty in the evidence can be used in situations where decisions for clinical practice and health policy have to be based on evidence that mainly comes from trials with indirect outcome measures.

Beyond assessing the quality of the evidence, potential benefits and harms of the intervention need to be weighed against each other and factors such as patient values, impact on healthcare equity, acceptability by patients and feasibility of the intervention need to be considered. The authors conclude that considering all these factors, a conditional recommendation for direct acting antiviral agents to treat chronic hepatitis C virus infection may be appropriate.


Reference: Dobler CC, Morgan RL, Falck-Ytter Y, Montori VM, Murad MH. Assessing the validity of surrogate endpoints in the context of a controversy about the measurement of effectiveness of hepatitis C virus treatment. BMJ evidence-based medicine 2018: 23(2): 50-53. https://ebm.bmj.com/content/22/6/199

Drug Value Symposium

The first Drug Value Symposium was held in Cleveland, Ohio on May 5th, 2018 in conjunction with the 8th Guideline Development Workshop using the GRADE approach.

Attendees found the inaugural Drug Value Symposium to be engaging, appreciating the opportunity to share ideas on how to systematically approach making evidence-based drug-coverage decisions at their institutions.

Attendees from all over the world with various backgrounds and expertise attended. The symposium opened with a thought-provoking presentation from Yngve Falck-Ytter, one of the founding U.S. GRADE Network members. His presentation highlighted why healthcare costs are becoming an increasing challenge and how incorporating a methodologically rigorous framework, such as GRADE, to formulary decision making can facilitate sensible decisions. 

Dr. Falck-Ytter opens the symposium with an overview of how GRADE includes both cost and cost-effectiveness when making health-care decisions.

Representatives from local healthcare systems then offered varying perspectives on the institutional successes and challenges of managing high cost drugs. Presenters included Mandy Leonard from Cleveland Clinic, Indrani Kar from University Hospitals, Christina Wadsworth and Jan Kover from MetroHealth, and Candice Wenzell from the Northeast Ohio VA Healthcare System. 

       Lessons learned from managing costly drugs within the Cleveland, OH 

Veteran's Administration, as presented by Ms. Wenzell.

The symposium concluded with an interactive session facilitated by Yngve Falck-Ytter, Candice Wenzell, and Kelsey Rife that allowed attendees to apply the GRADE methodology to evaluate a drug for a specific indication and then make a decision on whether or not they would recommend the use of the drug. 

Proposed new evidence-based medicine pyramid

From: MH Murad, N Asi, M Alsawas, F Alahdab. Perspective: New evidence pyramid. Evid Based Med. Online First: June 20, 2016. 

Systematic reviews and meta-analyses have been placed at the top of the evidence pyramid for several good reasons. They provide more trustworthy answers and more precise estimates with narrower confidence intervals that are were not selected based on expert opinion, but rather based on a systematic procedure. However, credible systematic review can summarize biased evidence and poorly done systematic reviews can summarize well done trials. This challenges the placement on top. In addition, GRADE tells us that our certainty in evidence should be driven by many factors other than study design. Therefore, we propose 2 modifications to the pyramid to resolve these 2 challenges.

Figure 1.

Legend: The proposed new evidence-based medicine pyramid. (A) The traditional pyramid. (B) Revising the pyramid: (1) lines separating the study designs become wavy (Grading of Recommendations Assessment, Development and Evaluation), (2) systematic reviews are ‘chopped off’ the pyramid. (C) The revised pyramid: systematic reviews are a lens through which evidence is viewed (and applied).

A year in review

A few highlights for the U.S. GRADE Network in 2015:

We hosted our second Guideline Development Workshop in partnership with U.S. Cochrane West (a branch of the U.S. Cochrane Center) in Portland, Oregon. The turnout was so large, it was hard to capture everyone in one photo! 

We hosted our third Guideline Development Workshop in Orlando, FL! Fun fact: The very first USGN Guideline Development Workshop was also held in Orlando in the fall of 2014. We had a great year and look forward to more activities in 2016!