GRADE Guideline Development Workshop - now with INGUIDE!

We at the U.S. GRADE Network are excited to announce that, for the first time ever, we have partnered with INGUIDE, a credentialing system for guideline developers, to offer free access to their Level 1 online training to participants of our Guideline Development Workshop and Guideline Fundamentals for Panel Members and Organization Staff pre-workshop course. 

This announcement comes just in time for our next workshop, held in Austin, Texas May 17-19, for which there are still spots available at our workshop website.

The INGUIDE program, the result of a collaboration between the Guidelines International Network (G-I-N) and McMaster University, provides certification of individuals involved in guideline development on four successive levels, from a Guideline Panel Member to becoming a Guideline Development Credentialing Instructor themselves. 

To obtain access to the INGUIDE Level 1 course, simply register for and attend our 3-day GRADE guideline development workshop in addition to the pre-workshop course on Guideline Fundamentals for Panel Members and Organization Staff. This 3-hour pre-workshop course will cover all of the necessary information about the practical aspects of guideline development, including:

• the importance of trustworthy guidelines
• checklists for the development of guidelines
• the structure and roles of a guideline panel
• the process of grading evidence and formulating recommendations

The main 3-day workshop will cover important aspects of assessing risk of bias, rating the certainty of evidence, and considering other important factors when using GRADE to develop a guideline.

Our pre-workshop course combined with our main three-day in-person course will provide all of the training required for a breeze-through experience online. You will receive free access to the ~2-3 hour online course, comprising four modules, that you can complete at your convenience after the workshop. Upon completion, you will be officially certified as an INGUIDE Level 1 Guideline Panel Member.

Why We Can't Wait for Our Austin Workshop

The U.S. GRADE Network's 18th workshop is just five weeks away! With a focus on assessing certainty of evidence informed by non-randomized studies, this workshop will take place in Austin, Texas - the home of Austin City Limits, the famous Bat Bridge, and some next-level barbecue joints.

We asked some of the workshop facilitators about what they're looking forward to about the trip (besides the chance to talk about GRADE and evidence-based medicine for three days straight!) Here's what they said...

Limited space remains for the workshop. Learn more and register and www.gradeconf.org. 

2022 Recap and What's Next for the U.S. GRADE Network

Thanks to you, 2022 was one of our best years yet.

We were relieved to see you in person, our first time since 2020, at our spring workshop in Chicago last June! Then, we reconvened online for a virtual workshop in November. Overall, we hosted over 100 participants from around the world, stretching from Canada and Spain to Colombia and Sweden. 

Facilitators and participants at the spring 2022 GRADE guideline development workshop in Chicago, IL - the first in-person USGN workshop since March 2020.

We also launched our new two-day virtual Systematic Review workshop offering and welcomed another 48 learners, including a record 13 scholars from across the globe!

We published our takeaways from our early experiences moving our workshop online in response to the Covid-19 pandemic in BMJ Evidence-Based Medicine, which you can read about here.

Finally, we launched the USGN Webinar Series and will host regular free webinars on all things GRADE. For our inaugural session, we were joined by Drs. Gordon Guyatt and Reem Mustafa. You can watch the recording and subscribe to our new YouTube channel here. 

We look forward to seeing you at our 18th Guideline Development Workshop, in person in sunny Austin, Texas, May 17-19, 2023! The workshop will have a special focus on assessing the certainty of evidence informed by non-randomized studies. Registration is now open.

From Risk of Bias to Living Evidence, Fall 2022 EF Scholars Present on Cutting-Edge EBM Projects

During our 17th GRADE Guideline Development Workshop, held virtually, the U.S. GRADE Network had the honor of introducing the three latest recipients of the Evidence Foundation scholarship to the virtual stage. On the third and final morning of the workshop, each scholar presented on their current projects to reduce bias in healthcare and move the field of evidence-based medicine methodology forward. 

Abrar Alshorman, MBBS, presented on her work to examine the tools used to assess risk of bias across various types of evidence. As choosing the wrong risk of bias tool at the start of an evidence synthesis effort might waste time and disrupt workflow, her project aims to guide systematic reviewers in choosing the best tool to use. The resulting list includes risk of bias tools for use in primary studies that were identified in a search of the evidence, including those with applications in randomized, non-randomized, diagnostic test accuracy, prognosis, and qualitative bodies of evidence. 

"The workshop struck the perfect balance between informative and interactive," said Alshorman. "[A]ll sessions have been extremely informative and provided by very knowledgeable presenters. This opportunity offered me insight into the key concepts of the GRADE approach for guideline development. I look forward to incorporating the information I've learned into practice."

Next, Brett Norling, a third-year medical student, discussed his project examining the use of the GRADE framework in systematic reviews in urology. The project reviewed literature published in the top 5 urology journals since 2000, using Perl programming code to identify articles using GRADE. The team then applied a 37-item checklist developed de novo from the "Criteria for Applying or Using GRADE." The results demonstrated that while most (95.8%) of the included reviews correctly applied GRADE on an outcome-level basis, nearly one-third (31.4%) of the 70 reviews summary of findings, evidence profile, or other from of table to present their findings.

"I came to the GRADE Workshop having performed a systematic review and a few other research projects regarding systematic review methodology," said Norling. "I felt that my understanding of GRADE was in the developing stages, though it felt a bit abstract prior to the workshop. Key GRADE principles of evidence assessment came to life as I worked with a small group to assess a body of evidence and arrive at a recommendation. I now feel well equipped to confidently assess evidence and perform systematic reviews with a greater level of autonomy than prior to this workshop."

Finally, Ariadna Alaudell-Rispau, a PhD candidate in the Biomedical Research Methodology and Public Health Doctoral Program at the Autonomous University of Barcelona, presented her proposal for the development of Living Evidence Structures Summaries (LESS). Auladell-Rispau described her project to lead a multi-design, iterative study that allows for a systematic and rigorous development of a "Living Evidence updates reporting model" that "accounts for the periodic and/or constant updates of the evidence syntheses developed under the Living Evidence model, permits the development of valid, reliable, and updated health information to health decision makers, including guideline or [health technology assessment] developers..., [and is] friendly and accessible." The project will include a systematic review, brainstorming meetings, a Delphi consensus process, and finally, testing and assessment of the newly developed model.

We look forward to updating the blog about these projects as they move forward!

ANNOUNCEMENT: Applications for the Spring 2023 GRADE Guideline Development Workshop, held in Austin, Texas, May 17-19, 2023, are due February 28th! Learn more about the application requirements here and read about the workshop details here.

COVID-END Working Groups Call for Living Systematic Reviews and Considerations of Health Equity in Evidence Synthesis and Guideline Efforts

The COVID-19 pandemic was not only associated with a rapid worldwide spread of a virus, but also of large amounts of information across the globe - not all of which was trustworthy or credible. Experts call this an "infodemic." In order to improve the synthesis and dissemination of trustworthy information in a manner that could keep up with the fast pace and ever-changing landscape of knowledge on COVID-19, the COVID-19 Evidence Network to support Decision-making (COVID-END) was established.

In a paper published in this month's issue of Journal of Clinical Epidemiology, McCaul and colleagues describe how the COVID-19 pandemic ushered in an urgent need to rapidly understand the etiology and management strategies for the disease, and to disseminate this information far and wide. However, a lack of collaboration resulting in duplication of work across institutions and countries hampered these efforts. COVID-END, comprising two working groups  dedicated to overseeing the coordination and dissemination of trustworthy evidence syntheses and guidelines, was a result of these unprecedented needs. The effort also included an Equity Task Group that evaluated the impact of evidence synthesis and recommendations on matters of health and socioeconomic disparities arising from or exacerbated by the pandemic.

Figure from McCaul et al. describing the efforts of COVID-END

The goal of the project, in the authors' words, was to support the "evidence supply side" by promoting already available resources and work led by institutions across the globe, both for those involved in evidence synthesis or the formulation of recommendations based on the evidence. The avoidance of effort duplication was highlighted by, for instance, urging guideline developers to first search for existing high-quality and up-to-date guidelines before beginning work on new recommendations. The development and use of living systematic reviews, which are continually updated as new evidence becomes available, is further highlighted as a way to improve the timeliness of evidence syntheses while reducing efforts put into new projects.

McCaul, M., Tovey, D., Young, T., et al. (2022). Resources supporting trustworthy, rapid and equitable evidence synthesis and guideline development: Results from the COVID-19 Evidence Network to support Decision-making (COVID-END). J Clin Epidemiol 151: 88-95. Manuscript available at publisher's website here. 

8 Steps Toward Incorporating Equity in Rapid Reviews and Guidelines

Along with the mass mobilization of systematic reviews of evidence ushered in by the COVID-19 pandemic was also the need to synthesize evidence and disseminate results as rapidly as possible. As part of the process of formulating guidelines based upon rapid reviews, the impact of decisions and policies on equity should be considered. In a newly published paper, Dewidar and colleagues provide specific steps for incorporating stakeholders and improving the consideration of equity in the context of rapid guidelines.

The project was part of work conducted by the Equity Task Force of the global COVID-19 Evidence Network to support Decision-making COVID-END) network. The team was diverse in terms of the gender (70% women), regions (17% from low-middle-income countries), and career stages (40% early career) represented. The resulting guidance was created in line with the steps outlined in the Cochrane Handbook's chapter on equity and followed the PRISMA-Equity (PRISMA-E) extension for reporting. The team then identified published systematic reviews related to COVID-19 that focused on populations experiencing inequities as categorized by the PROGRESS-Plus paradigm - for instance, by Place of Residence (health systems in rural areas and their preparedness for outbreaks), Education (the impact of educational attainment on adherence to COVID-19 health guidelines), and Disability (the impact of COVID-19 on those with disabilities) - for examples that review authors can incorporate equity into their own reviews.

The authors conclude that greater involvement of diverse stakeholders can encourage the consideration of more diverse social factors in the development and interpretation of systematic reviews and resulting guidelines and policies. Rapid reviews also benefit from having a translation plan that includes methods for disseminating findings in a way that is consistent with the goal of reducing inequities. 

Dewidar, O., Kawala, B.A., Antequera, A., et al. (2022). Methodological guidance for incorporating equity when informing rapid-policy and guideline development. J Clin Epidemiol 150: 142-153. Manuscript available at the publisher's website here. 

U.S. GRADE Network Describes Experience Moving to All-Virtual Workshop Format

On March 4-6, 2020, the U.S. GRADE Network held an in-person workshop in Phoenix, Arizona, much like any of the 11 workshops to have come before it. Participants and facilitators enjoyed a taco buffet bar together at the first night's reception, sat together in small and large rooms to learn and collaborate, and mingled over coffee and pastry refreshments during breaks.

One week later, the World Health Organization announced that COVID-19 had reached pandemic proportions. 

Over that summer, the USGN took our workshops online, hosting three consecutive fully virtual workshops in October 2020, May 2021, and October 2021. While some changes were made (the addition of multiple, 45-60 minute breaks, for instance, to accommodate eating times in multiple time zones), much of what lay at the heart of a GRADE workshop remained: a three-day format  including plenary lectures from PICOs to recommendations, presentations by Evidence Foundation scholars, and small-group, hands-on experiential learning opportunities.

The USGN's shift to an all-virtual setting, and its challenges as well as opportunities for growth, are presented in a new paper by Siedler and colleagues published online in the BMJ Evidence-Based Medicine journal. Using routine feedback survey data collected both before and after the pandemic, the authors (all GRADE workshop facilitators) found that...

• Perceived understanding of GRADE improved to the same extent in virtual and in-person formats,
• At least half of attendees (54-62%) indicated that the virtual format was important for their ability to attend, and
• Participants indicated a high degree of workshop satisfaction and perceived educational value. Similar results were observed for the level of knowledgeability of speakers, value of plenary sessions, and helpfulness of small-group sessions.

The major takeaway from the USGN's experience in an all-virtual format is that, based upon positive feedback and the ability to reach a global audience of learners, it will continue to offer learning opportunities in a virtual setting this year and beyond.

In fact, the next all-virtual workshop will take place November 30-December 2, 2022, and registration is now open at www.gradeconf.org.

Siedler MR, Murad MH, Morgan RL, et al. (2022). Proof of concept: All-virtual guideline development workshops using GRADE during the COVID-19 pandemic. BMJ Evidence-Based Medicine (online before print). Manuscript available from publisher's website here.