Training

The most critical training consideration for guideline development is understanding how to make recommendations using the information at hand.

Watch Dr Sue Brennan, Director of the Melbourne GRADE centre and Associate Professor Zachary Munn, Director of the Adelaide GRADE Centre talk about the use of GRADE in guidelines.

Training

Print this page

The NHMRC Standards reflect the need for transparency, the need for a mix of expertise and experience in the guideline development group, and the need to be evidence-informed. The development of research questions and the review of evidence by the guideline development group is a complex process. Your guideline group may need some additional training to work together through this process.

The training needs of individual members should be assessed before the guideline development group meets for the first time. You will need to be particularly sensitive to the needs of members who have not been involved in guideline development or research before.

NHMRC recommends GRADE (Grading of Recommendations, Assessment, Development and Evaluation) for guideline development, particularly for those guideline developers seeking NHMRC approval. The most critical training consideration for guideline development is understanding the process for prioritising outcomes, reviewing the evidence and making recommendations based on the evidence. GRADE provides a structured and systematic approach to undertake these tasks.

One-day courses to learn about GRADE in guideline development are available through the two Australian accredited GRADE training centres, the Melbourne GRADE Centre and the Adelaide GRADE Centre.

By identifying needs and providing training for individual members, you will be creating the best conditions for group members to contribute equally, and for their contribution to be given appropriate consideration during group discussions, decision-making, and when the group is formulating recommendations.

This module provides an outline of common training needs for members of a guideline development group to help them effectively produce credible and trustworthy guidelines.

What to do

1. Consider the overall development approach of the guideline

Whether you are developing a new guideline, updating an existing guideline or adapting a guideline, you need to prepare members of the guideline development group for the approach you are planning to use. Training will be dependent on who you are involving in the development process, and whether you need members to be proficient in particular steps of the guideline development process, such as an ability to review and interpret health economics or modelling data.

You need to make decisions about what training can be conducted in-house and what needs to be outsourced. These decisions will have implications for how you allocate resources for the project and will need to be accounted for in the budget and project plan.

Options to consider include:

training in evidence review skills to review and interpret data and research in order to make recommendations
training in the process used to make recommendations, for example training in the use of the GRADE framework for moving from the evidence to a decision (EtD)
media training to communicate key messages to the public, particularly if the guideline will be controversial and likely to attract media attention
communications training, such as assistance with writing the guideline or speaking about the guideline
application training, for example in the use of MAGICapp, GRADEProGDT or other applications that use GRADE’s evidence-to-decision framework for the development of guidelines using those programs.

Your approach to stakeholder consultation may require training in:

facilitation skills, particularly if you plan to run focus groups
survey design, to create and distribute surveys for target groups.

2. Consider the needs of the chair

Chairs are critical to facilitating group discussion, and guiding committees through the process for making recommendations so it is important that they are experienced with or trained in this area. NICE has summarised content that should be covered when inducting a guideline committee chair, which includes the guideline development process, forming the questions and reviewing evidence, developing and wording recommendations, principles of facilitation, declaring interests and dealing with conflicts of interest. Please also see the Guideline development group, Identifying and managing conflicts of interest, Forming the questions, Identifying the evidence modules.

If your chair will be the spokesperson for the guideline and there is a possibility they will be conducting interviews or talking on behalf of the guideline development group, you should consider training in media skills.

3. Consider the needs of individual guideline development group members

Your guideline development group members will have been chosen for their knowledge, expertise, or their ability to consult with a specific sector. Some members may have already been recruited with pre-requisite guideline development skills, but it is also likely you will have members who are new to guideline development or perhaps unfamiliar with current guideline development approaches. It is important to make yourself available to answer newer members’ questions so they feel confident in undertaking tasks, particularly at the beginning of the process.

Once members are appointed, it is prudent to talk to them individually about specific areas of the guideline development process where they might like additional training. Important aspects of the process that members may need to be familiar with include:

an overview of GRADE in guideline development
formulating and developing clinical questions using frameworks like PICO, identifying and prioritising outcomes that are important to consumers of health care
the GRADE approach for assessing the certainty of evidence
presenting evidence summary tables
making recommendations using an evidence to decision (EtD) framework and assigning a ‘strength of recommendation’ using standard terminology

There are a number of useful resources on these topics available at https://cebgrade.mcmaster.ca/.

You may identify other training requirements for members depending on the role they are playing on the development group. For instance, training in the writing of clear, jargon free plain-English may be necessary if the group members will be writing parts of the guideline.

G-I-N (Guidelines International Network) has a toolkit for patient and public involvement in guidelines, which provides practical advice on best practice, pitfalls to avoid and different methods of involvement including inclusion in development groups or consultation processes (Guidelines International Network (G-I-N) Working Group 2012).

4. Prepare introductory materials about the guideline development process

Guideline development group members will need to understand the relevant policy and procedures of your organisation, particularly disclosure of interest policies, confidentiality requirements, code of conduct and any administrative or reporting requirements.

They will also need information specific to the project, such as meeting attendance, key project personnel and contacts, the project plan, the terms of reference, and expectations you have about the role they will play as part of the guideline development group.

You should provide this package of information to all members either prior to, or at, the first meeting as an induction package. This could include guideline development methods — for example, a brief overview of the GRADE approach; a glossary of key terms; diagrams that represent the EtD process; or marked-up examples of evidence profiles to orient members to the structure and content.

5. Run an induction session

Allocate time at the first meeting to discuss the induction package and what training is available to help familiarise members to the guideline development process. This is a very important step in helping guideline development group members to orient themselves to the project.

An induction session should be used to discuss the overall approach you are planning to take with the guideline, including what aspects of it will be outsourced and what members will be expected to do. It is a good idea to invite a methodologist or systematic reviewer to give an overview of the evidence review process, including forming the questions, searching and evaluating the evidence, interpreting data and drafting the recommendations.

As each group tends to apply EtD frameworks differently, the induction session should introduce the EtD process to the guideline development group as a whole, given group dynamics and a shared understanding of the process can have a big impact on the process of moving from evidence to recommendations.

Demonstrating the EtD process with a mock panel exercise can be an effective way for the panel to learn the process of making recommendations and address any problems with interpreting the evidence summaries or wording of the strength of recommendations. It can also help establish good group processes that: encourage consumer participation; facilitate more efficient meetings; promote better compliance with GRADE methods and reduce the risk of having to revisit recommendations after decisions have been made.

6. Consider training members to use guideline development technologies

There are a number of guideline development platforms developed to help the decision-making process of guideline development groups — GRADEPro GDT and MAGICapp are modelled on the GRADE process and are the most widely-used.

GRADEPro GDT is a web-based tool for summarising and grading evidence in GRADE table formats (evidence profiles and summary of findings tables), and for making recommendations using the EtD framework. The EtD includes prompts to help structure and document the considered research evidence, additional considerations and judgements transparently.

MAGICapp is an open-source tool that helps guideline developers to collaboratively develop, write and publish guidelines in a digital environment using the GRADE approach (MAGIC 2019).

NHMRC requirements

Guidelines approved by NHMRC must meet all requirements as outlined in the Procedures and requirements for meeting the NHMRC Standard. The following requirements are relevant to the Training module:

A.3 A multidisciplinary group that includes end-users, relevant disciplines and clinical experts is convened to develop the purposes, scope and content of the guideline, and the process and criteria for selecting members are described.
A.4 Consumers participate in the guideline development, and the processes employed to recruit, involve and support consumer participants are described.
Desirable A.4.1 The guideline development process includes participation by representatives of Aboriginal and Torres Strait Islander peoples and culturally and linguistically diverse communities (as appropriate to the clinical need and context), and the processes employed to recruit, involve and support these participants are described.

NHMRC Standards

The standards relevant to the Training module are:

Standard 3 — Be overseen by a guideline development group
3. The guideline development group will:: 3.1. Be composed of an appropriate mix of expertise and experience, including relevant end users; 3.2. Have clearly defined, documented processes for reaching consensus

The composition of the guideline development group should reflect the range of individuals and organisations whose activities, services or care will be covered by the guideline (NICE 2014).

Standard 6 — Be evidence informed
6. To be evidence informed guidelines will:: 6.1. Be informed by well conducted systematic reviews; 6.2 Consider the body of evidence for each outcome (including the quality of that evidence) and other factors that influence the process of making recommendations including benefits and harms, values and preferences, resource use and acceptability.