Guidelines for Guidelines

Overview

The recommendations of a guideline are the key actionable statements that, if implemented appropriately, will achieve the objectives of the guideline¹. This may be a recommendation to do something (e.g. wash your hands), to stop something (e.g. de-prescribing a medication) or in some cases to choose the option preferred by those affected by the recommendation (where there is a close balance between pros and cons of options). For most decision-makers, having recommendations from a group who can systematically consider the evidence and that is well placed to make judgements, is preferable to having no recommendation at all^2,3.

Systematic reviews can provide a comprehensive summary of the evidence addressing each guideline question but they do not include recommendations. Many factors must be considered when making a recommendation, and the systematic judgement of the guideline development group is crucial in this process. Following a structured process, guideline development groups must weigh up the evidence and other relevant information to decide on the recommended course of action. 

Guideline development groups can find the process of moving from the evidence to a recommendation challenging, especially if the evidence base is uncertain or direct evidence is lacking. Given the many factors that can influence a recommendation, it is important that guideline development groups use a predetermined framework that enables a structured and transparent process to make decisions. A summary of the evidence, the information considered, and the judgments made should be published alongside each recommendation⁴.

‘Evidence to Decision’ (EtD) frameworks (arising from the GRADE Working Group and related work, and the work of the World Health Organisation (WHO) and the National Institute for Health and Care Excellence (NICE) were developed to help guideline development groups use evidence to make recommendations in a systematic and transparent way. These frameworks prompt the guideline development group to consider and document all relevant factors (criteria) and their judgements in order to make recommendations. The factors considered include whether the desirable effects of an option outweigh the undesirable effects (i.e. benefits versus harms), how much people value the main outcomes of an intervention or exposure, the feasibility and acceptability of different options, the impact on health equity, and resource requirements. 

EtD frameworks have been tailored for diverse types of evidence and guideline questions. Some agencies have developed EtD frameworks that are structured to support their own norms and values such as the WHO-integrate framework. Given NHMRC advises developers to use the GRADE framework, this module will focus on those elements covered in GRADE EtD frameworks which are available for making decisions about interventions^5,6, diagnostic tests⁷, coverage decisions⁸, health system and public health decisions⁵, and environmental and occupational health⁹.

GRADE EtD frameworks are a good practice model, developed through an iterative process and evaluated in a wide range of guidelines internationally^1,10. A key feature of the EtD frameworks is that they include criteria shown in existing guidelines to be important for determining the direction or strength of recommendations. The criteria are deliberately broad to allow guideline developers to focus on factors that are most relevant to their guideline. The widespread use of GRADE EtD frameworks internationally means that they provide a familiar format and standard terminology, making guidelines easier for decision-makers to use. 

For those planning to adopt or adapt recommendations from an existing guideline, it is still necessary to follow an EtD process, even if the evidence synthesis is current and relevant and the GRADE-ADOLOPMENT methods assists guidelines developers to do this^11,12 (see the Adopt, adapt or start from scratch module). Doing so is particularly important if an EtD process was not used in the existing guideline or if there are contextual differences that need to be considered. Several approaches are possible, the choice of which involve balancing the need for context-specific judgements in the new guideline with efficiency and resource constraints¹³.

What to do

EtD frameworks were intended to help structure the process by which recommendations are made in guideline meetings. The steps guideline groups can take to prepare for and implement the EtD process are described in the sections that follow and outlined in a checklist appended to this module. For simplicity, GRADE terminology is used throughout because this is the most widely used framework. Software platforms like MAGICapp and GRADEpro provide templates for preparing and publishing GRADE EtD frameworks. The software includes functionality to facilitate decision-making at different stages of the development process, including voting ahead of and during meetings. If there is a circumstance where GRADE or current frameworks are not appropriate for your guideline, it is still important to plan and use a structured, explicit and transparent approach when making recommendations. The factors considered in your decision making, judgements made for each and the evidence supporting the judgements must be reported.

1. Set up an EtD framework to document discussion

As with the other methods used in the guideline development process, the framework used to go from the evidence to making a recommendation should be planned in advance and outlined in a protocol. At the planning stage, guideline developers need to set up EtD framework templates suitable for their guideline questions. This involves deciding on the factors (criteria) to include in the EtD framework and what evidence will be sought to inform judgement about each criterion. The first step is to identify the perspective that will be taken when making recommendations and any groups or settings for which decisions may differ.

Identify the perspective
GRADE has EtD templates tailored to the different perspectives that recommendations can be made from. A guideline panel making recommendations to help an individual make decisions about their care will need to take a slightly different perspective from that for a public health, health systems or other population-level recommendation. For example, at population-level resource requirements (costs or savings) of a recommendation can be an important consideration, whereas individuals may only be concerned about out-of-pocket costs. Guideline development groups will need a clear idea of the perspective they are taking in order to select a template that prompts them to consider the factors relevant to their recommendations. This should be stated in the background to the guideline because the same perspective is usually taken for all recommendations.

Identify groups or settings for which decisions may differ
Guideline development groups need to consider whether there are specific considerations for certain populations or settings that could lead to different judgements. This has implications for the way the EtD framework is set up, especially when the judgements lead to a different recommendation, so should be considered at an early stage. In the Australian guidelines to reduce health risks from drinking alcohol, separate recommendations were made for the general population, women who are pregnant, and young people. To facilitate the panel’s decisions and reporting, a separate EtD framework was set up for each population. In contrast, in the Australian Living Guideline for the Pharmacological Management of Inflammatory Arthritis, a single recommendation was made to not routinely use opioids for the treatment of pain in rheumatoid arthritis, with a sub-recommendation for those with severe pain for whom other analgesic options had failed. A single EtD was used, and specific populations (such as those at risk of increased harm) were addressed in the practical information supporting the recommendation. In making their judgements, the guideline group also noted that the balance of benefits and harms might be altered for people without access to comprehensive pain management services. Different perspectives or judgements can be identified through:

• the guideline development group
• separate panels set up to inform a specific perspective (for example the EAC Working Group for Aboriginal and Torres Strait Islander Women’s Antenatal Care for the Pregnancy Care Guidelines), or 
• specific targeted consultation with particular groups. 

It is important to plan for how these perspectives will be presented in the EtD framework so that they can be integrated in the GDGs deliberations.

Decide on the factors (criteria) to consider
GRADE EtD frameworks are intended to be flexible, rather than a rigid tool that the GDG must adhere to. The criteria in GRADE EtD frameworks are deliberately broad to allow guideline developers to focus on factors that are most relevant to their guideline. Table 1 shows the standard criteria considered in a GRADE EtD for making recommendations about interventions, and the evidence that can be used to judge each criterion. Similar criteria are considered for making recommendations about diagnostic tests, screening and prognosis^6,14. 

Guideline development groups will need to consider whether all criteria are relevant to the recommendations they intend to make. Factors that are genuinely not relevant or are outside the remit of a guideline group may be removed. On the other hand, some GRADE criteria cover factors that are of sufficient importance for certain decisions that they should be considered separately. For example, guideline developers making recommendations about psychotropic medicines may include legal constraints as a separate criterion to ensure consent requirements are considered¹⁵.There may be additional unique criteria that need to be considered by the guideline development group. It is important that these are specified in your template. For instance, community voice and cultural considerations were included in the EtD framework used to develop the Recommendations for culturally safe and clinical kidney care for First Nations Australians. WHO have integrated human rights and sociocultural acceptability into a framework (WHO-INTEGRATE) that is fit for purpose for WHO public health guidelines^16,17. GRADEpro software enables guideline developers to use standard templates for different perspectives (including the WHO Etd), or customise templates by selecting or adding criteria. If you are using templates in software like MAGICapp you can use subheadings within the EtD template to differentiate factors that are important to your decision making.

Decide on what evidence will be sought to inform judgements for each criterion
NHMRC Standard 6 requires guidelines to be evidence informed. For questions about the desirable and undesirable effects of an intervention on health outcomes (benefit and harms), it is best practice to use well-conducted systematic reviews as the source of evidence. Similar expectations apply to questions about prognosis and the accuracy of diagnostic, screening and other tests. Systematic reviews can also be used to inform judgements for most other criteria, however the extent to which this is done varies. In many guidelines, developers take a pragmatic approach such as using a systematic review or study identified through an ad hoc process. Other sources of evidence are listed in Table 1. Those adopting or adapting existing recommendations may use evidence presented in the EtD from the source guideline(s), alongside contextual evidence from local studies or solicited from the GDG¹³.

Equity-related evidence is a special case. Equity can be addressed systematically for each EtD criterion (Box 1)^26,27. Doing so may enable better integration of equity considerations in deliberations²⁸.

For all criteria, additional considerations can inform judgements¹⁴. This may include local data, collected from administrative sources or specifically for the guideline. Such data may help understand equity implications, the acceptability of options, and factors that influence the feasibility of implementing recommendations. It may also include what the GDG considers to be plausible reasons for why options may not be acceptable, feasible or would lead to inequity. Panel members will bring knowledge and experience that should be captured in the EtD if it influences the recommendation. Methods have been developed for systematically gathering this ‘expert evidence’ from consumers and health professionals^29,30.

2. Decide on the process for using the EtD to make recommendations

The EtD framework is a tool for group-decision making, facilitating structured discussion and explicit judgements about the evidence considered when making a recommendation. Guideline groups vary in how they use the EtD to make recommendations. When the EtD is fully implemented, the GDG makes judgements for each EtD criterion based on discussion of the evidence and additional considerations, and then decides on the recommendation. In more streamlined processes, some steps are undertaken by the technical team that prepares the evidence or by a smaller working group comprised of members of the GDG (e.g. co-chairs). The process used will depend largely on the preferences and circumstances of the guideline development group. You do however have to document the method you used to finalise the recommendations. 

Planning who will be responsible for each step, and how decisions will be made, can ensure that the implications of different approaches are understood. The process used may influence the quality of decisions, have resource implications, and influence the time required of GDG members ahead of and at meetings.

Decide on the process for discussing evidence and making judgments about each criterion
The process used to make these decisions should capture the views of all GDG members, and surface divergent perspectives and interpretations of the evidence about each criterion³¹. These are essential inputs for informing the recommendation. Ideally, GDG members will have opportunity to make initial judgements prior to hearing other perspectives, so that the full diversity of views is captured. There should then be opportunity for GDG members to consider the perspectives of other members and reach a decision, using a collaborative approach that supports balanced representation of different interests.

A possible process that involves iterations of feedback is as follows:
Ahead of the GDG meeting

• Circulate the draft EtD to GDG members 
• GDG members review the evidence and make an independent judgement for each EtD criterion (i.e. unaware of decisions made by other members) 
• GDG members have the option to provide comments and additional considerations 

At the GDG meeting

• Briefly present the evidence and comments/additional considerations 
• Present the judgements for each EtD criterion
• Discuss points of disagreement identified from the judgments or comments, encouraging those with divergent views to share the reasons for their judgement 
• Poll GDG members privately to determine if there is sufficient consensus to proceed to the next EtD criterion or recommendation 

At this stage, polling is used to gauge agreement and identify where to focus discussion. Polling can be a fast way to give all members an equal say and gauge their views. Both GRADEpro and MAGICapp have survey functions that enable distribution of the draft EtD and voting (polling) on each EtD criterion. Results can be shared at meetings (anonymously, if preferred) and polling repeated to support iterative decision-making.

In streamlined processes, the EtD may include a proposed judgement on which the GDG’s agreement is sought. This approach is more common when a recommendation from an existing guideline is considered for adoption or adaption¹³.

Decide on the process for agreeing on the direction and strength of recommendations 
The judgements made for the EtD criteria should guide the GDG’s decision-making, but a process will be needed to agree on the direction and strength of the recommendation (see Section 5 for explanation of types of recommendations). In some cases, the EtD judgements will clearly favour one option. Where there is a close balance, the GDG must consider the implications and importance of each of the EtD judgments. Note that the importance of EtD criteria may vary across recommendations.

Typically, the GDG will vote on the direction and strength of a recommendation, although some groups use more informal approaches to reach consensus. Voting may be anonymous or non-anonymous. As with EtD decisions, a recommendation may be proposed by the technical team or a subgroup of the GDG and the GDG’s agreement with the proposal sought.

Irrespective of whether voting is used, recommendations should be made through consensus decision-making whereby the aim is to achieve consent from all members³¹. It is not a requirement in GRADE or the NHMRC standards to achieve 100% agreement on a recommendation; however, the proportion who disagree should be recorded alongside reasons for the opposing views. What counts as consensus should be predefined (e.g. unanimity, supermajority based on a specified decision threshold, simply majority). Using decision thresholds for consensus rather than seeking unanimity may enable the GDG to better reflect the diversity of views. The WHO handbook provides a more detailed critique of decision-making methods and decision-rules for reaching consensus³¹.

Decide on the process for writing recommendations
As with the preceding steps, it is sensible to decide who will draft the recommendation, and whether this will be done ahead of, during or after the guideline meeting. Some groups prefer not to pre-empt the GDG decision on a recommendation, whereas others find that providing a draft recommendation can help the panel reach consensus on direction and strength, particularly for more borderline decisions. The precise wording of recommendations may need to be developed and agreed after the meeting. 

In addition to the recommendation (i.e. the actionable statement), health professionals and others will need practical information to apply the recommendation (see for example the ‘practical information’ tab in a  recommendation on the use of opioids for pain in rheumatoid arthritis). Discussions at the meeting can inform the content of this practical information, so it may be helpful to plan for how this can be addressed at meetings.

3. Draft content for the EtD framework(s) with a summary of evidence for each question

At this stage of guideline development, the EtD framework is a working document intended to inform and capture panel discussions and decision-making. The draft EtD framework should include a brief summary of research evidence and additional information for the GDG to consider. It may also include questions (or prompts) to elicit the GDG perspectives and experience during the GDG meeting¹⁴. 

Prepare a summary of findings for each question addressed by systematic review, reporting certainty of the evidence for critical and important outcomes
Standard tables are available in MAGICapp and GRADEpro for summarising findings from systematic reviews of intervention effects. These concise, structured summaries (or ‘evidence profiles’) are designed to support decision-making at GDG meetings. The results in the evidence profile can be briefly presented at a GDG meeting, with the aim of reaching agreement on the interpretation of the effects of the intervention on each outcome.

The evidence profile should include results for all outcomes agreed as critical and important for making the recommendation, irrespective of whether evidence was found. A GRADE plain language statement can be used to interpret each result (implemented in MAGICapp and GRADEpro)³². The statements provide standardised phrasing to describe the effect on the outcome (beneficial or harmful), the size of the effect (e.g. large enough to be of importance to patients or not), and how certain the evidence is (e.g. “Perioperative discontinuation of DMARDs may increase the risk of flare”). Using these phrases helps the GDG agree on and communicate their interpretation of findings. Details on how to prepare an evidence profile are in the module on Assessing certainty of evidence. For an example, see the ‘research evidence’ tab for recommendations on second generation antipsychotics for people living with dementia.

The EtD framework includes a section for summarising the desirable (beneficial) and undesirable (harmful) effects. This section should address the outcomes presented in the evidence profile, but can also include additional considerations. A version of the evidence profile can be inserted into the EtD (an option in GRADEpro or word formats). Alternatively, the EtD summary may be based on the plain language interpretation of each result (see Box 2). This latter approach may help the GDG make their judgement about how substantial the overall desirable and undesirable effects are (i.e. trivial, small, moderate, large) if they have previously agreed on thresholds for interpreting the effect estimates and assessing certainty of the evidence. Ideally, thresholds will have been agreed after rating the importance of outcomes but before reviewing evidence³³. The overall certainty of evidence is reported separately in the evidence profile.

Prepare a concise summary of evidence and additional considerations for other EtD criteria
GDG groups vary in the extent to which they have evidence and additional considerations to inform other EtD criteria. While systematic reviews provide the best evidence for most criteria, it is common that pragmatic approaches are used. Nonetheless, it is important to summarise any information that the GDG should consider and include prompts to capture their perspectives during discussion. The summary should be designed to support decision-making at GDG meetings.

If you have results from an evidence synthesis for other EtD factors, GRADE summary of findings table templates are available for evidence about values and cost effectiveness^19,34,35. There is an online tool for preparing a summary of findings from a qualitative evidence synthesis in which GRADE CERQual has been used: iSoQ. This may be useful for summarising a synthesis of evidence about equity implications, acceptability or feasibility of intervention options.

Link to more detailed information as required
The content of an EtD is usually based on more detailed evidence and other information. It is preferable to link to supporting information, rather than present lengthy text in the EtD itself. Both MAGICapp and GRADEpro have the option to link out to documents and websites.

Examples of more detailed information to which links can be provided include: 

• full systematic reviews (prepared by the technical team or existing reviews) 
• tables reporting characteristics and methodological quality of studies 
• forest plots and results from additional analyses 
• overarching documents examining equity or implementation considerations for the guideline as a whole 
• data from surveys of GDG members, advisory groups or other interest holds 
• related clinical practice guidelines or resources. 

4. Use the EtD framework to structure discussion and decide whether the net consequences favour the intervention or the alternative

To make a recommendation, the GDG must decide whether the evidence and additional information favours the intervention, favours the alternative option (the comparator), indicates a close balance between options, or is uncertain. All EtD criteria are usually considered when deciding on ‘net consequences’, not just the desirable and undesirable effects on health outcomes.

Discuss and make a judgement about whether the balance between the desirable and undesirable effects (‘net benefit’) favours the intervention or the alternative option
A useful first step is for the GDG to review the content of the evidence profile and agree on the interpretation of each outcome. Once agreement is reached on the interpretation of each outcome, the panel must make a judgement about whether the desirable effects of the option being considered outweigh the undesirable effects.

This judgement requires the GDG to consider: 

1. How substantial are the desirable anticipated effects of the intervention option compared to the alternative (trivial, small, moderate or large)? 
2. How substantial are the undesirable effects (trivial, small, moderate or large)? 
3. What is the overall certainty of evidence of effects (generally the lowest certainty on any of the outcomes deemed to be critical for decision-making) 
4. Is there important uncertainty or variability in how people value the main outcomes (e.g. would some accept the small risk of serious side-effects for a moderate reduction in their pain, whereas others would not) 

The GDG then makes an overall judgement about the balance between the desirable and undesirable effects (Table 2 shows judgements implemented in GRADEpro and MAGICapp). If the balance clearly favours one option over the other, this judgement is likely to be instrumental in determining the direction of the recommendation. More often, there will be trade-offs because the option has both desirable and undesirable effects⁴.

^*Note that the wording in MAGICapp implies that the option to be recommended is known (‘of the recommended alternative’). This differs from GRADEpro, where the responses lead to a recommendation.

Discuss and make a judgement about whether other criteria (impact on resources and equity, acceptability, feasibility) favour the intervention or the alternative
In some instances, the recommendation will be clear from the net effects and other EtD criteria need not be considered. For instance, where there is high certainty evidence of large undesirable effects (harm) the GDG need not consider other criteria to recommend against the intervention. Most often, the panel will consider all criteria in their framework, although some EtD criteria may be more important than others for a particular recommendation. Slightly different response options are used to judge the other EtD criteria, but all judgments are intended to help the GDG decide whether the evidence and other considerations favour the intervention or the alternative option.

Record each judgement and the basis for the judgement
During the meeting, each judgment made by the GDG and the factors that influenced the judgements should be recorded. Where there is substantive disagreement about the evidence, the different viewpoints should be recorded even if consensus was subsequently reached. If additional considerations raised during the meeting influenced the GDG judgements, these considerations should be documented in the EtD.

5. Make clear, concise and actionable recommendation(s) and/or good practice statements

As a final step in the GDG meetings, the group should use the judgements made in their EtD framework to decide on their recommendation. The GDG may also decide to include good practice statements (GPS) in the guideline. As with recommendations, these actionable statements should be developed through a systematic and transparent process. What differentiates these two types of actionable statements is that recommendations are based on systematic review and assessment of the certainty of evidence (for desirable and undesirable effects at a minimum) whereas good practice statements are not. Specific criteria must be met for a GPS (described below).

Weigh up the judgements across the EtD criteria to decide on the direction and strength of the recommendation
Formal recommendations should provide “an actionable statement about the choice between two or more management or policy options (interventions) in a specific population and, if relevant, in a specific setting”³⁶. It follows that each recommendation has a direction - the GDG must recommend for or against the intervention. GRADE recommendations have one of two strengths, strong or conditional. Table 3 shows the interpretation of strong and conditional recommendations, and the circumstances in which each can be used^4,37. 

Different terminology has been used for conditional recommendations (including ‘weak’), however conditional conveys that the recommendation is subject to one or more conditions that may be unrelated to certainty in the evidence. For example, a conditional recommendation would be warranted when we have high certainty in the benefit and harms of an option, but know that some patients would decline the option whereas other would not. Strong recommendations are mainly used where the desirable consequences clearly outweigh undesirable; however, there are some exceptions such as where there is uncertain benefit but certain harm³⁷.

Other recommendations implemented in MAGICapp are in Table 3, including public health recommendations, statutory requirements and research only recommendations. 

GRADEpro includes an overview of judgements that the GDG may find helps guide their overall judgement. Having deliberated over the body of evidence for each criterion, the overall judgement can be relatively straightforward.

Write the recommendation as a concise, clear and actionable statement
Recommendations should be specific and practical with a concise, clear and actionable statement of what needs to be done. Table 3 contains some examples of recommendations from NHMRC approved guidelines (for others, see MAGICapp and https://guidelines.gradepro.org/search).

The recommendation must: 

• state what the action is and be limited to one main action
• have a clear direction (for or against the option in the recommendation question)
• state the population and setting (if relevant) 
• state the comparator (unless the alternative option is obvious) 
• include sufficient detail for those responsible to carry out the action (do not assume the action is self-explanatory or the steps are obvious)

In addition, when writing recommendations:

• use consistent words and phrasing to convey the strength of the recommendation (for example, NICE in the UK uses the word ‘offer’ for strong recommendations and ‘consider’ for conditional³⁸.
• consider using symbols and words for strength to avoid misinterpretation
• write in the active voice wherever possible
• avoid using acronyms, jargon, abbreviations and unnecessary technical language
• use shorter, plain English words over longer forms and minimise punctuation

Choose action-orientated verbs to convey the strength and urgency of a recommendation. For example, “listen for rhonchi”, “remove dressing after three days”, “stop blood transfusion immediately”, “eat a variety of nutritious foods from the five food groups” and “issue a boiled water advisory.” Resist the temptation to vary the verbs you use simply to avoid repetition. Guidelines that have a consistent nomenclature are often far easier to navigate. They are also more suitable for electronic implementation - for example decision support or automated data linkage.

Provide a justification (rationale) for the recommendation based on how the guideline group weighed the criteria
The final EtD must include a justification (rationale) for the recommendation made by the GDG. The justification should convey how the panel weighed up evidence and other information, focusing on what drove their decision. It is not a restatement of the evidence (this is summarised within the EtD), rather it should convey the GDG’s overall interpretation of the evidence, focusing on final deliberations and any points of disagreement. An example is provided below on tranexamic acid.

Use the EtD process to develop good practice statements
In certain circumstances, a GDG may develop actionable statements without a systematic review of evidence. GRADE provides criteria and a proposed process for developing these good practice statements (GPS) (Box 3).

If the criteria for a GPS are met (Box 3), then it is inappropriate to use GRADE methods for developing a formal recommendation³⁹. The certainty of evidence is not graded and the GPS must be clearly labelled as a ‘good practice statement’ (not a ‘recommendation’). The converse is also true, a question addressed by systematic review should be developed as a formal recommendation using GRADE methods even if only very low certainty indirect evidence was found⁴⁰. An actionable statement developed after systematic review should not be presented as a good practice statement (or other type of ‘statement’) simply because the evidence is of low or very low certainty. 

The basis for a good practice statement can vary. Table 4 summarises the requirements with examples³⁹.

^†Based on Dewidar, O., Lotfi, T., Langendam, MW., Parmelli, E., Saz Parkinson, Z., Solo, K., et al. (2022) Good or best practice statements: proposal for the operationalisation and implementation of GRADE guidance. BMJ Evidence-Based Medicine, doi:10.1136/bmjebm-2022-111962.

^‡Taxonomy based on Norris, SL. (2024) GRADE Good Practice Statements - A time to say Good-bye? A new typology for normative statements on interventions. J Clin Epidemiol, 171:111371. doi: 10.1016/j.jclinepi.2024.111371, and Dewidar, O., Lotfi, T., Langendam, MW., Parmelli, E., Saz Parkinson, Z., Solo, K., et al. (2022) Good or best practice statements: proposal for the operationalisation and implementation of GRADE guidance. BMJ Evidence-Based Medicine, doi:10.1136/bmjebm-2022-111962.

6. Finalise, review and publish the EtD content and recommendation(s)

The final version of the EtD should provide a complete and transparent account of the GDG decisions and the basis for those decisions. In addition to meeting NHMRC requirements, the GDG should ensure that requirements for reporting recommendations in their guidelines are met⁴¹.

Finalise the wording of each recommendation (including remarks) and any practical information needed to apply the recommendation
Most GDGs will agree on the direction and strength of a recommendation in meetings, but the wording of the recommendation usually needs to be finalised after the meeting. 

Those using the guideline will also need practical information to help them implement the recommendation. MAGICapp includes a section (tab) for ‘practical information’ related to each recommendation and there are many good examples on the platform (for example, see the accompanying practical information for the recommendations in Table 4). This practical information is sometimes referred to as ‘implementation guidance’, ‘implementation considerations, tools and tips’ or the ‘how, who, where, what and when’ for applying the recommendation^36,40. This practical information is intended to support the recommendation (it does not ‘stand-alone’). The GDG has complete discretion to include whatever content they consider important for the recommendation, but this can include dose and duration of treatment, how to monitor and discontinue treatment, considerations for particular groups of patients, and links to external resources (including other guidelines) and support services. 

Any specific considerations for particular population groups (sometimes referred to as ‘subgroups’) should be summarised, especially if the recommendation is conditional on population-related characteristics.

Finalise the EtD content and the justification (rationale) for each recommendation, ensuring these provide a transparent account of all judgements
The final EtD should include explicit and direct links to the evidence that underpinned each decision. Any new evidence or additional considerations that was influential should be addressed in the framework (directly or via linked sources). For each EtD criterion, the GDGs judgement should be presented. It should be clear how the GDG weighed up the evidence to reach these judgements and the overall decision about the direction and strength of the recommendation. 

Some EtD templates, such as those in GRADEpro, also have sections that address implementation (e.g. facilitators of equitable access, health professional skill development), monitoring and evaluation, and research priorities.

Have the GDG review the draft recommendations and supporting information (including EtDs)
It is critical that the GDG has opportunity to review and comment on any changes to the EtD, practical information, and in particular the recommendation prior to publication. At a minimum, all GDG members should approve the final wording of the recommendation.

References

1. Rosenbaum, SE., Moberg, J., et al. (2018) Developing Evidence to Decision Frameworks and an Interactive Evidence to Decision Tool for Making and Using Decisions and Recommendations in Health Care. Global Challenges, 2, 1700081. https://doi.org/10.1002/gch2.201700081
2. Neumann, I., Alonso-Coello P., et al. (2018) Do clinicians want recommendations? A multicenter study comparing evidence summaries with and without GRADE recommendations. J Clin Epidemiol, Jul:99:33-40. doi:10.1016/j.jclinepi.2018.02.026 
3. Neumann, I. and Schünemann, HJ. (2020) Guideline groups should make recommendations even if the evidence is considered insufficient. CMAJ, Jan 13;192(2):E23-E24. doi: 10.1503/cmaj.190144.
4. Andrews, J., Guyatt, G., Oxman, AD., Alderson, P., Dahm, P., Falck-Ytter, Y., et al. (2013) GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations. J Clin Epidemiol, 66:719-25. doi: 10.1016/j.jclinepi.2012.03.013
5. Moberg, J., Oxman, AD., et al. (2018) The GRADE Evidence to Decision (EtD) framework for health system and public health decisions. Health Res Policy Syst, May 29;16(1):45. doi: 10.1186/s12961-018-0320-2.
6. Alonso-Coello, P., Schünemann, HJ., Moberg, J., Brignardello-Petersen, R., Akl, EA., Davoli, M., et al. (2016) GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ, 353. doi:10.1136/bmj.i2016
7. Schünemann, HJ., Mustafa, R., et al. (2016) GRADE Guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health. J Clin Epidemiol, Aug:76:89-98. doi: 10.1016/j.jclinepi.2016.01.032.
8. Parmelli, E., Amato, L., et al. (2017) GRADE EVIDENCE TO DECISION (EtD) FRAMEWORK FOR COVERAGE DECISIONS. Int J Technol Assess Health Care, Jan;33(2):176-182. doi: 10.1017/S0266462317000447.
9. Senerth, E., Whaley, P., Akl, EA., et al. (2024) GRADE Guidance 40: the GRADE evidence-to-decision framework for environmental and occupational health. ESS Open Archive. May 10. doi:10.22541/essoar.171535972.23271792/v1
10. Neumann, I., Santesso, N., et al. (2016) A guide for health professionals to interpret and use recommendations in guidelines developed with the GRADE approach. J Clin Epidemiol, 72:45-55. doi: 10.1016/j.jclinepi.2015.11.017.
11. Klugar, M., Lotfi, T., Darzi, AJ., Reinap, M., Klugarová, J., Kantorová, L., et al. (2024) GRADE Guidance 39: Using GRADE-ADOLOPMENT to adopt, adapt or create contextualized recommendations from source guidelines and evidence syntheses. J Clin. Epidemiol. 174;111494. https://doi.org/10.1016/j.jclinepi.2024.111494 
12. World Health Organization. Regional Office for E. (2023) Strengthening countries’ capacities to adopt and adapt evidence-based guidelines: a handbook for guideline contextualization. Copenhagen: World Health Organization. https://iris.who.int/handle/10665/372275 
13. Kahale, LA., Ouertatani, H., Brahem, AB., Grati, H., Hamouda, MB., Saz-Parkinson, Z., et al. (2021) Contextual differences considered in the Tunisian ADOLOPMENT of the European guidelines on breast cancer screening. Health Research Policy and Systems, 19:80. https://doi.org/10.1186/s12961-021-00731-z
14. Alonso-Coello. P., Oxman, AD., Moberg, J., Brignardello-Petersen, R., Akl, EA., Davoli, M., et al. (2016) GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ, 353. doi:10.1136/bmj.i2089 
15. Hilmer, SN., Gnjidic, D., Reeve, E., Kalisch, Ellett, L., Wu, H., Raymond, J. (2019) Use of Antipsychotic Medicines for People aged 65 years and over: A review undertaken for the Australian Commission on Safety and Quality in Health Care. Northern Sydney Local Health District, St Leonards NSW, Australia. 
16. Norris, SL., Rehfuess, EA., et al. (2019) Complex health interventions in complex systems: improving the process and methods for evidence-informed health decisions. BMJ Global Health, 4:e000963. https://doi.org/10.1136/bmjgh-2018-000963
17. Rehfuess, AE., Stratil, JM., et al. (2019) The WHO-INTEGRATE evidence to decision framework version 1.0: integrating WHO norms and values and a complexity perspective. BMJ Glob Health, Jan 25;4(Suppl 1):e000844. doi: 10.1136/bmjgh-2018-000844. 
18. Zeng, L., Helsingen, LM., Bretthauer, M., Agoritsas, T., Vandvik, PO., Mustafa, RA., et al. (2023) A novel framework for incorporating patient values and preferences in making guideline recommendations: guideline panel surveys. J Clin Epidemiol, 161;164-172. https://www.jclinepi.com/article/S0895-4356(23)00175-0/fulltext
19. Zhang, Y., Li, S-A., Yepes-Nuñez, JJ., Morgan, RL., Pardo-Hernandez, H., Alonso-Coello, P., et al. (2022) GRADE summary of findings tables enhanced understanding of values and preferences evidence. J Clin Epidemiol, 147:60-8. doi:10.1016/j.jclinepi.2022.03.007 
20. Brunetti, M., Shemilt, I., Pregno, S., Vale, L., Oxman, AD., Lord, J., et al. (2013) GRADE guidelines: 10. Considering resource use and rating the quality of economic evidence. J Clin Epidemiol, 66(2):140-50. https://doi.org/10.1016/j.jclinepi.2012.04.01 
21. Akl, EA., Welch, V., Pottie, K., Eslava-Schmalbach, J., Darzi, A., Sola, I., et al. (2017) GRADE Equity Guidelines 2: Considering health equity in GRADE guideline development: Equity extension of the Guideline Development Checklist. J Clin Epidemiol, 90;68-75. doi:10.1016/j.jclinepi.2017.01
22. Welch, VA., Akl, EA., Pottie, K., Ansari, MT., Briel, M., Christensen, R., et al. (2017) GRADE equity guidelines 3: health equity considerations in rating the certainty of synthesized evidence. J Clin Epidemiol, 90:76-83. doi:10.1016/j.jclinepi.2017.01.015
23. Lewin, S. and Glenton, C. (2018) Are we entering a new era for qualitative research? Using qualitative evidence to support guidance and guideline development by the World Health Organization. International Journal for Equity in Health, 17(1):126. doi:10.1186/s12939-018-0841-x 
24. Lewin, S., Glenton, C., Lawrie, TA., Downe, S., Finlayson, KW., Rosenbaum, S., Barreix, M., Tunçalp, Ö. (2019) Qualitative Evidence Synthesis (QES) for Guidelines: Paper 2 - Using qualitative evidence synthesis findings to inform evidence-to-decision frameworks and recommendations. Health Res Policy Syst, 17:75. https://doi.org/10.1186/s12961-019-0468-4 
25. Glenton, C., Lewin, S., Lawrie, TA., Barreix, M., Downe, S., Finlayson, KW., Tamrat, T., Rosenbaum, S., Tuncalp, Ö. (2019) Qualitative Evidence Synthesis (QES) for Guidelines: Paper 3 - Using qualitative evidence syntheses to develop implementation considerations and inform implementation processes. Health Res Policy Syst, 17:74. https://doi.org/10.1186/s12961-019-0450-1 
26. Lin, JS., Webber, EM., Bean, SI., Evans, CV. (2024) Development of a Health Equity Framework for the US Preventive Services Task Force. JAMA Netw Open, 7:e241875. doi: 10.1001/jamanetworkopen.2024.1875. 
27. Pottie, K., Welch, V., Morton, R., Akl, EA., Eslava-Schmalbach, JH., Katikireddi, V., et al. (2017) GRADE equity guidelines 4: considering health equity in GRADE guideline development: evidence to decision process. J Clin Epidemiol, 90:84-91. doi: 10.1016/j.jclinepi.2017.08.001. 
28. Barnabe, C., Pianarosa, E., Hazlewood, G. (2021) Informing the GRADE evidence to decision process with health equity considerations: demonstration from the Canadian rheumatoid arthritis care context. J Clin Epidemiol, 138:147-55. https://doi.org/10.1016/j.jclinepi.2021.06.013 
29. Schünemann, HJ., Zhang, Y., Oxman, AD. (2019) Distinguishing opinion from evidence in guidelines. BMJ, 366:l4606. https://doi.org/10.1136/bmj.l4606
30. Mustafa, RA., Garcia, CAC., Bhatt, M., Riva, JJ., Vesely, S., Wiercioch, W., et al. (2021) GRADE notes: How to use GRADE when there is "no" evidence? A case study of the expert evidence approach. J Clin Epidemiol, 137:231-5. doi:10.1016/j.jclinepi.2021.02.026. 
31. World Health Organisation (2014) WHO handbook for guideline development, 2nd Ed, 16: Decision-making for guideline development at WHO. https://iris.who.int/handle/10665/145714
32. Santesso, N., Glenton, C., Dahm, P., Garner, P., Akl, EA., Alper, B., et al. (2020) GRADE guidelines 26: informative statements to communicate the findings of systematic reviews of interventions. J Clin Epidemiol, 119:126-35. doi:10.1016/j.jclinepi.2019.10.014.  
33. Schünemann, HJ., Neumann, I., Hultcrantz, M., Brignardello-Petersen, R., Zeng, L., Murad, MH., et al. (2022) GRADE guidance 35: update on rating imprecision for assessing contextualized certainty of evidence and making decisions. J Clin Epidemiol, 150:225-42. doi:10.1016/j.jclinepi.2022.07.015.
34. Xie, F., Shemilt, I., Vale, L., Ruiz, F., Drummond, MF., Lord, J., et al. (2023) GRADE guidance 23: considering cost-effectiveness evidence in moving from evidence to health-related recommendations. J Clin Epidemiol, 162:135-44. doi:10.1016/j.jclinepi.2023.08.001.  
35. Lewin, S., Bohren, M., Rashidian, A., Munthe-Kaas, H., Glenton, C., Colvin, CJ., et al. (2018) Applying GRADE-CERQual to qualitative evidence synthesis findings-paper 2: how to make an overall CERQual assessment of confidence and create a Summary of Qualitative Findings table. Implement Sci, 13:10. https://doi.org/10.1186/s13012-017-0689-2 
36. Lotfi, T., Hajizadeh, A., Moja, L., Akl, E., Piggott, T., Kredo, T., et al. (2022) A taxonomy and framework for identifying and developing actionable statements in guidelines suggests avoiding informal recommendations. J Clin Epidemiol, 141:161-71. doi:10.1016/j.jclinepi.2021.09.028.
37. Andrews, JC., Schünemann, HJ., Oxman, AD., Pottie, K., Meerpohl, JJ., Coello, PA., et al. (2013) GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation's direction and strength. J Clin Epidemiol, Jul;66:726-35. doi:10.1016/j.jclinepi.2013.02.003.
38. National Institute for Health and Care Excellence (NICE). (2014) Developing NICE guidelines: the manual. nice.org.uk/process/pmg20
39. Dewidar, O., Lotfi, T., Langendam, MW., Parmelli, E., Saz Parkinson, Z., Solo, K., et al. (2022) Good or best practice statements: proposal for the operationalisation and implementation of GRADE guidance. BMJ Evidence-Based Medicine, doi:10.1136/bmjebm-2022-111962.
40. Norris, SL. (2024) GRADE Good Practice Statements - A time to say Good-bye? A new typology for normative statements on interventions. J Clin Epidemiol, 171:111371. doi: 10.1016/j.jclinepi.2024.111371.
41. Chen, Y., Yang, K., Marušić, A., et al. (2017) A Reporting Tool for Practice Guidelines in Health Care: The RIGHT Statement[J]. Annals of Internal Medicine, 166(2):128-132. doi:10.7326/M16-1565.  

Acknowledgements

NHMRC would like to acknowledge and thank Dr Sue Brennan from Cochrane Australia who authored this module.