Guidelines for Guidelines

Overview

While considerable resources are used in the development of guidelines, often less attention is paid to determining how and when a guideline will be updated¹ . Maintaining a guideline (especially ensuring it is based on up-to-date evidence) is critical for the ongoing relevance, reliability and trustworthiness of guideline recommendations.

Keeping guidelines current, particularly in areas where there is significant research activity or change in the healthcare context, requires sustained funding and expertise not only to monitor the evidence but to also make decisions on when and how guideline recommendations are maintained.
It is common that even when the need for an update has been established, uncertainty about funding means initiation of the update process is delayed until funding can be secured. Despite this it is still important to document when an update to a guideline is required even if there is some delay before work on the update begins.

Another complicating factor is that recommendations in a guideline will need updating with varying frequency, requiring different monitoring schedules or review dates. To manage this effectively requires a shift in focus from updating a full guideline to updating individual recommendations and assigning a review date to each recommendation. The review date will be dependent on a number of factors and is relevant for traditional guidelines (where for instance it might be reviewed every five years)² or living guidelines (for instance being reviewed within the year)³.

Living guidelines differ in that they are designed to continually monitor and rapidly incorporate new evidence, often using automated systematic review methods⁴. This may involve real time updating mechanisms, such as linking specific recommendations to relevant ongoing clinical trials to ensure they reflect the most current and relevant advice. The Australian Living Evidence Collaboration is driving these efforts in Australia and has published several guidelines under this model. The Living Guidelines Handbook outlines these methods in detail.

Whether your guideline is traditional or living, priority should be given to monitoring recommendations based on clinical or stakeholder need, and identifying ongoing research or other changes in the healthcare context that may impact the recommendations.

It is the responsibility of a guideline development group to establish and justify what monitoring approach is warranted (an ideal plan) as the guidelines are being developed, even if funding is not secured for the update. This includes outlining the priorities for review, setting the thresholds for when a recommendation should be reviewed and changed and to set a review date for individual recommendations (see Figure 1). Being transparent about the decisions and priorities at the time can enhance the ‘updatability’ of guidelines for future iterations even if it is a different group or funder that is responsible for the next update or reestablishing priorities.

Figure 1. Steps in updating a recommendation

What to do

1. Decide in advance what circumstances will trigger an update

Shekelle et al have suggested 6 situations that indicate when a guideline needs updating⁵:

1. Changes in the evidence on the existing benefits and harms of interventions
2. Changes in the outcomes that are considered important
3. Changes to the available interventions
4. Changes to the evidence that current practice is optimal
5. Changes in the values placed on outcomes
6. Changes in the resources available for health care

As you are developing a recommendation, it is important to consider what could trigger the need to update it and when it could potentially occur. Some of these factors could already be known (such as the publication of an upcoming large trial that is likely to have a substantial impact upon practice) or could occur in the future (such as emerging adverse events from post-market surveillance of a new therapy).

Triggers can be content/situation based or time based. In addition to the above 6 situations, other likely scenarios that could trigger an update include:

• new information, emerging evidence or new therapies that have substantial implications for recommendations
• the addition of new evidence to the body of evidence results in a change to the conclusions of syntheses of that evidence, and a likelihood that the recommendations could then change
• priority needs expressed by guideline users or funders
• the age of the guideline (e.g. after 3 or 5 years)
• changes in the nature of a disease (new variants of COVID, clades of mpox, or changes in prevalence of illness)
• changes in the health system such as scope of practice changes
• policy changes (e.g. a medication receives government funding, or roll-out of a new screening program)

Triggers for update and how monitoring and review would occur could be documented in a specific guideline section (i.e. updating or maintaining currency) or outlined in the discussion text for specific recommendations.

Table 1 outlines how to consider the evidence base and types of recommendations you are developing and what impact that will have on monitoring and review.

Table 1. Consideration of the evidence and recommendations and impact on monitoring and review

Evidence base example
Established evidence that has high certainty and unlikely to change or inactive research topics	Moderately active evidence that is building, changing certainty or could be evolving to challenge current practice; limited or uncertain evidence with new data expected	Active body of evidence being rapidly published but still uncertain
Type of recommendation example
A principle-based recommendation	An area of interest to certain groups and regulators	An emerging area of significant interest to consumers/populations
Topic example
Recommendations around the delivery of care	Disinfection methods for water treatment	COVID therapies
Review triggers or thresholds example
Changes in population demography Changes to eligibility of a screening program Care program no longer offered/available	Longitudinal cohort studies release new data New technologies become available or more cost effective to use	New therapies are rapidly being developed and tested in an active research program
Monitoring examples
Journal alerts	Post-market surveillance from manufacturers	Actively running systematic searches in databases
Review frequency example
Between 5-10 years	Watching brief – 3-5 years	As new evidence is published or at least yearly

2. Decide how you will monitor what circumstances will trigger an update

To determine what circumstances will trigger an update there needs to be a system in place to monitor activity and gather feedback once the guideline is published⁶.

Active monitoring requires a team of people who can apply the appropriate tools for surveillance, review new evidence and process feedback, and an oversight group to make decisions on any required changes to the guidelines in a responsive timeframe.

Passive monitoring (for example relying on users to contact you or setting up journal or media alerts) still requires people who can receive information and act on it as appropriate but, because this may not require attention regularly, it is far less resource intensive.

3. Document a planned approach for the next guideline update

Once you have determined circumstances that could trigger an update and how they could be monitored, it is important to describe a basic conceptual approach to the next update for the guideline.

In Garner et al’s framework⁷ to assess systematic reviews they’ve outlined the following questions to consider when planning an update:

• Assess currency
  • Are the questions still relevant?
  • Has there been good uptake?
  • Are the methods of the evidence review still valid?
• Identify new methods/studies/information
  • Are there new relevant methods?
  • Are there new studies or other information?
• Assess the effect of updating the guidelines and whether the additional work is justifiable
  • Will new methods change the findings, quality, reliability or credibility?
  • Will new studies or data change the findings, quality, reliability or credibility?

These questions can be used to guide initial discussions with funders regarding the scope and cost of an update and as a starting point for the review. To address some of these questions you will likely need input from experts.

Table 2 outlines some considerations that should be described in a planned approach to update the guideline.

Once funding is secured and the project is ready to commence, the plan for updating follows a similar plan to that for developing a guideline (see the Project planning module) where detailed scoping, budgets and resource allocation are put in place. The plan for updating should include provision for conducting a public consultation process for any changes to the wording, strength or direction of recommendations.

The CheckUp checklist⁸ can assist in the planning of an update approach.

4. Decide on the extent of the update

It is more than likely you may only have resources, or it may only be appropriate, to undertake a partial update or an update relevant to specific topics. In this case you still need to document the decisions regarding what sections will be updated and why.

Refreshed guideline (no change to recommendations)

There may be supporting content or information regarding implementation that could be modified (e.g. GRADE Evidence to Decision information), but the recommendations themselves are unlikely to change. In this case you should make the amendments and outline this in a change log. You should also mark the decision for when the next major update should occur. The date of the last evidence search should still be retained to ensure there is a transparent link between when the evidence was last reviewed for the specific recommendation. If there are no changes to the recommendations but the recommendations and content have been reviewed, this would be considered a refreshed guideline.

Partial update to a guideline

If only certain recommendation statements need to be updated, or if newly available treatments emerge that require new consideration, this would be considered a partial update. With a partial update it should be made clear what new recommendations have been added and what has not changed since the guideline was last published.
Sometimes there is a long delay between the release of a guideline and the next update. In these circumstances you would need to reassess the scope and consider:

• whether the recommendations are still relevant in current practice (e.g. are the interventions still in use or is the research area still active)
• whether the guideline is still being used, and whether it had an impact when it was first released
• if the original guideline was of sufficient quality, in light of current standards.

Full guideline update

Where many recommendations are out of date, making the entire guideline invalid, there would need to be a full update. This will require the previous guideline to be withdrawn and formally rescinded.

NHMRC’s 2020 Australian Guidelines to Reduce Health Risks from Drinking Alcohol and the process to update the Australian Dietary Guidelines (currently under review) are good examples of reestablishing priorities relevant to stakeholder need and developing new methods as part of the updating process.

5. Prioritise topics within the update

If a guideline is large in scope and/or has many recommendations it may not be feasible to update every topic concurrently. Consider allocating a prioritisation system, such as high, medium and low priority to each recommendation and explain the prioritisation strategy to users. This could be based on:

• how fast the evidence is changing
• stakeholder need and/or interest
• the likelihood that new evidence will affect the recommendations
• the extent to which the recommendation or new information has a substantial impact on consumers, stakeholders or the health system
• whether new studies/information/data will change findings or credibility

Consider what action needs to be taken based on the level of priority. For example, a low priority rating may indicate that a search is run once a year for that topic whereas a high priority rating may have alerts set up for new publications or studies to be reviewed as soon as they are published.

Circumstances where you might decide to assign a low priority to a recommendation or delay updating the evidence base underpinning a recommendation include:

• potentially relevant studies ongoing but not complete
• the certainty of evidence underpinning the recommendation is high
• the recommendation is principle based but essential for good practice
• new information is identified but it is unlikely to change the review findings.

For example, this principle- based recommendation from the Australian Pregnancy Care Guidelines is unlikely to need regular review and so would be considered a low review priority.

In contrast Australian recommendations for prevention of communicable disease will require close adherence to advice in the Australian Immunisation Handbook, which is updated regularly in response to surveillance of new and emerging vaccines and indications.

6. Ensure the date of the last review is marked against recommendations

There are two parts to a guideline update: one is the need to keep the evidence on which a recommendation is based up-to-date, and the second is updating the recommendation itself. In this case there may be instances where the evidence is monitored and reviewed but it does not result in a change to the recommendations. If this is the case it is important that the date of last review is clearly marked to show that the recommendation is based on the most current body of evidence⁹, or that information is provided on how it should be used.

Therefore each recommendation within a guideline needs to clearly articulate:

• the last date evidence was reviewed (including the search dates of evidence)
• whether any new relevant information was identified and its impact on the recommendation, including supporting information
• when the recommendation will likely next be reviewed (assigning a “use by” or “will be updated by” date to the recommendation).

Software like MAGICapp is able to store versions of recommendations to help users track the history of updates. Developers may often use terms like ‘this recommendation was updated on [search date x], and ‘no new studies were identified’ or ‘one new study was identified that resulted in no change in the strength or direction of the recommendation’ to indicate the outcome of a search on the recommendation.

The incorporation of new evidence over time can be documented in a PRISMA flow chart. An example of this is the PRISMA flowchart from the International evidence-based guideline for the assessment and management of polycystic ovary syndrome (PCOS) which can be found in the accompanying technical reports.

7. Be transparent about what has been updated

Changes that you have made to your guideline need to be communicated to users of the guideline especially if only certain topics have been updated. At a minimum, a section that details what has changed from the previous version should be published with the new version, including the rationale for the changes.

Once you have published the guideline you can make editorial adjustments and minor clarifications to text as necessary, but this should be considered as guideline maintenance - not updating. If minor changes have been made ensure this is documented in an activity or change log that is published alongside the guideline. An example of this is the change log in the 2019 Australian Guidelines for the Prevention and Control of Infection in Healthcare on MAGICapp. Importantly, the date of the official publication of the guideline should still be adhered to when referencing the guideline but it should be made clear when the individual recommendation was last reviewed.

Guideline recommendations should not be changed under maintenance activity. If changes to recommendations are required, an updating process needs to be initiated that involves the consideration of new evidence by a multidisciplinary expert group and consultation on these recommendations.

Once you update your guideline, you should ensure that older versions are clearly identified as outdated. If this is difficult, for instance if there are numerous printed versions in circulation, there should be a strategy in place to promote the current version and communicate to users how to manage older versions. This is an important step in publication governance that is often overlooked, and has the risk of exposing end users of guidelines to incorrect and out of date guidance.

With frequent updating of recommendations or transitioning traditional guidelines to living guidelines it can be hard for a user to track what has been updated and why. Having a clear statement and rationale in your guideline that outlines not only what has been updated but what content has remained the same or been modified slightly and why.

The Australian Pregnancy Care Guidelines are an example of detailing how topics are updated and viewed in historical context alongside more recent recommendations.

NHMRC publications policy

NHMRC’s publication policy is that all documents be reviewed after 5 years, from the date of publication. After this time, they can be reviewed and re-issued (starting another five-year cycle). If this does not occur, NHMRC approval is withdrawn (NHMRC Council’s 148th session report June 2003). Publications more than 10 years old are advised to be rescinded, unless there are good reasons for individual documents to be retained.

A rescindment notice is displayed inside the front cover of an NHMRC Council approved publication that is older than five years. The text used is outlined in the box below.

Under the NHMRC Approval Program, third party guidelines can receive NHMRC approval for up to five years. This approval period only applies to the version submitted at the time of the approval. Any subsequent versions and/or updates to the guideline recommendations require the guideline to be resubmitted for approval.

Useful Resources

A useful checklist that outlines what procedures should be published when guidelines are being updated is outlined at Reporting Items for Updated Clinical Guidelines: Checklist for the Reporting of Updated Guidelines (CheckUp) | PLOS Medicine. 

Standards for planning and conduct of UPDATES of Cochrane Intervention Reviews (U1-U11)

G-I-N McMaster Guideline Development Checklist 

G-I-N Updating Guidelines Working Group

National Institute for Health and Care Excellence (NICE): The guidelines manual Chapter 14 Updating published clinical practice guidelines and correcting errors

World Health Organization. Regional Office for Europe. (2023). Strengthening countries’ capacities to adopt and adapt evidence-based guidelines: a handbook for guideline contextualization. World Health Organization. Regional Office for Europe. https://iris.who.int/handle/10665/372275. License: CC BY-NC-SA 3.0 IGOIOM updating advice

Covidence

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) Checklist

Centre for Reviews and Dissemination’s (CRD’s) guidance for undertaking reviews in health care

References

1. Shekelle, P., et al. (2001) When should clinical guidelines be updated? BMJ, 2001. 323(7305): p. 155-7. doi: 10.1136/bmj.323.7305.155 
2. Alderson, LJ., Alderson, P., Tan, T. (2014) Median life span of a cohort of National Institute for Health and Care Excellence clinical guidelines was about 60 months. J Clin Epidemiol. 2014 Jan;67(1):52-5. doi: 10.1016/j.jclinepi.2013.07.012.
3. Akl, EA., et al. (2017) Living Systematic Review Network. Living systematic reviews: 4. Living guideline recommendations. J Clin Epidemiol. 2017 Nov;91:47-53. doi: 10.1016/j.jclinepi.2017.08.009.
4. Cheyne, S., et al. (2023) Methods for living guidelines: early guidance based on practical experience. Paper 1: Introduction. J Clin Epidemiol, 2023. 155: p. 84-96. doi: 10.1016/j.jclinepi.2022.12.024.
5. Shekelle, P., et al. (2012) Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development. Implement Sci, 2012. 7: p. 62. doi: 10.1186/1748-5908-7-62.
6. Garcia, LM, et al. (2012) Strategies for monitoring and updating clinical practice guidelines: a systematic review. Implementation Science, 2012. 7: p. 109. https://doi.org/10.1186/1748-5908-7-109 
7. Garner, P., et al. (2016) When and how to update systematic reviews: consensus and checklist. BMJ, 2016. 354: p. i3507. https://doi.org/10.1136/bmj.i3507 
8. Vernooij, R.W., et al. (2017) Reporting Items for Updated Clinical Guidelines: Checklist for the Reporting of Updated Guidelines (CheckUp). PLoS Med, 2017. 14(1): p. e1002207. https://doi.org/10.1371/journal.pmed.1002207
9. Atkins, B., et al. (2023) Improving prioritization processes for clinical practice guidelines: new methods and an evaluation from the National Heart Foundation of Australia. Health Res Policy Syst, 2023. 21(1): p. 26. https://doi.org/10.1186/s12961-022-00953-9  

Acknowledgements

NHMRC would like to acknowledge and thank Dr Saskia Cheyne, Dr Tari Turner and Heath White from the Australian Living Evidence Collaboration, for their contribution to the development of this module.