Guidelines for Guidelines

Guidelines have the potential to affect a wide range of people and organisations by influencing behaviour, changing practice and impacting policy, legal or funding frameworks. Stakeholders are individuals or groups who are affected by or can influence a guideline’s development and implementation (IOM 2011), including:

• government agencies
• professional societies and the members they represent
• people affected by the condition (as individuals and collectively in consumer organisations)
• healthcare and academic centres
• quality improvement organisations
• relevant commercial companies
• other end users.

Stakeholders include ‘expected users’—those who you might expect or intend to apply, implement, or be otherwise directly affected by your guideline recommendations. In this module, stakeholders are classified as anyone with a legitimate interest in your guideline or anyone affected by its recommendations (see Table 1 for examples).

A compelling reason to engage stakeholders is that it can enhance the implementation of your guideline (Grol, Dalhuijsen et al. 1998). First, a guideline developed with input from stakeholders is more likely to be relevant to the needs of its expected users, which improves its implementability. Second, early involvement by organisations that oversee the administration and funding of services that will use your guideline can lead to greater support for its implementation. These benefits are more likely to happen if stakeholders are engaged early in the development process using a consultative approach (Jones 2011).

You should engage stakeholders at critical points during the development process, such as when the guideline is scoped and when the outcomes are prioritised. While a variety of engagement methods can be used, evidence of their benefits and limitations in guideline development comes from case studies of the experience of developers rather than comparative evaluations. As with all aspects of guideline development, your choice of methods and how you apply them should be balanced with available resources and other constraints (Institute of Medicine 2011) .

The following principles (adapted from Department of Health, 2015) can be used to plan authentic stakeholder engagement:

• plan in advance for when and how to engage stakeholders
• involve stakeholders from the start
• set clear expectations by explaining what their role is and how their contribution will inform the guideline
• make it easy for stakeholders to be involved, particularly those who may find participation more difficult because of language, culture, age, disability or financial and other barriers
• be aware of stakeholder objectives, expertise and level of influence
• acknowledge and respect stakeholder views
• treat stakeholders equitably.

Some stakeholders, such as consumers, should be involved throughout all stages of guideline development—this advice is discussed in other modules (see Consumer involvement, Guideline Development Group and Public Consultation). The focus of this module is on the general methods of engaging stakeholders throughout guideline development.

What to do

1. Plan for early and repeated stakeholder involvement

Stakeholder perspectives can be considered at different steps in the guideline development process and should be planned in advance.

Some agencies use topics suggested by stakeholders to commission and prioritise new guidelines (NICE 2014; Shin 2014; SIGN 2015). Early involvement might include:

• proposing or commenting on guideline topics
• the relative importance of outcomes
• the settings and expected users of the guideline (see Guideline Development Group, Scoping the guideline and Considering purpose and context).

You should provide clear opportunities for the involvement of appropriate stakeholders. These may include reviewing recommendations and derivative products to ensure clear and actionable messaging and planning for implementation (Cluzeau, Wedzicha et al. 2012).

It is important that developers of Australian guidelines involve Aboriginal and Torres Strait Islander Peoples as members of the guideline development group wherever possible, particularly if key outcomes differ in these populations. If this is not possible then you will need to consider and document alternative ways to seek their views as stakeholders. For example, developers of the NHMRC Borderline Personality Disorder guidelines wrote:

Although the Guideline Development Committee membership did not include representation from Aboriginal and Torres Strait Islander peoples, the views of this community were canvassed early in the process through consultation with the Department of Health and Ageing’s Aboriginal and Torres Strait Islander Committee.

Once your guideline has been implemented, stakeholders might contact you with feedback about its usability and relevance. They may also let you know when updates are needed, such as when new evidence becomes available, or when changes in practice or policy occur (SIGN 2015).

While involving stakeholders can boost the relevance and use of your guideline, it can also add to the overall development time and may increase your administrative burden depending on the engagement methods you choose to use (Cluzeau, Wedzicha et al. 2012). This will present a particular challenge if you are developing guidelines with a volunteer workforce, or if you need to develop guidelines quickly, so you should plan accordingly based on your available resources and deadlines (see the Project planning module).

2. Identify local and international stakeholders

Begin this process by conducting a stakeholder analysis. This will help you to determine who should be considered when developing your guideline by systematically gathering and evaluating qualitative information (Schmeer 2001).

First define the expected users of your guideline by thinking about who might use or be affected by its recommendations (Canadian Taskforce on Preventive Health Care, 2017). For example, a guideline on the treatment of muscle strains may be developed for physiotherapists, but other expected users might include general practitioners, primary care nurses and people with muscle strains.

Expected users are only a subset of stakeholders. Other interested parties in guideline development that can also be considered stakeholders include funding bodies and industry groups. Understanding this distinction will help you later when planning how to involve different stakeholders, for instance when deciding which groups should be involved in decisions about your guideline and which others should simply be informed about its development and progress.

It is important to identify not only local stakeholders but also international groups that are developing or publishing advice on similar guideline topics. Searching government websites such as Australian Clinical Practice Guidelines, National Institute for Health and Care Excellence (NICE) Guidance (UK) and Centers for Disease Control and Prevention (CDC) (USA) may help you find guidelines in development or published and it may be helpful to contact their developers for advice on your guideline’s scope and selection of topics.

It is better to be non-selective at this stage to ensure that you identify stakeholders across a wide range of relevant groups, regardless of their interests (see Table 1 for examples of types of stakeholders).

3. Consider levels of involvement for different stakeholders

Once you have identified relevant stakeholders, consider setting up a database and recording stakeholder details such as their:

• name
• organisation
• position
• contact details
• location.

It is also useful to record their type—for example, whether they are from the government or private sector, or represent a condition group or professional society.

An important step in determining appropriate levels of involvement is to list all possible stakeholders and consider their degree of relevance to the guideline (Table 2). Relevance in this case refers to how great an impact the guideline may have on them. It can also refer to their capacity to affect, positively or negatively, the development or implementation of the guideline. For example, if you are developing public and environmental health guidelines you should contact and stay in touch with relevant government jurisdictions. You should also align with their advice, since they will likely be ultimately responsible for implementing your guideline.

While not all stakeholders need to be directly engaged, it is important that you consider:

• who might be interested in your guideline
• what their views might be
• what influence they might have on different stages of development.

While the degree of relevance can be a useful measure when determining levels of involvement, it is also important that you consider the interests of all stakeholders and whether these might preclude them from particular engagement activities.

These are important decisions for you to make as the content and implementation of guidelines may change as a result of stakeholder perspectives and whether they are convergent or contrasting (Green and Aarons 2011).

4. Decide how and when to engage stakeholders

When you are deciding the timing and methods for involving stakeholders you should consider at which stages of development it will be most important to seek their input and plan for distinct engagement activities (Cottrell, Whitlock et al. 2014). You should:

• decide on an appropriate level of engagement (see Section 3)
• let the focus and desired outcome of the engagement inform the method of choice
• avoid tokenistic involvement
• develop a communications plan which specifies the aim, method and logistics of each engagement activity (see Table 3 for examples of methods).

For all methods it is important that you clearly define which aspects of the guideline are the focus of engagement. Prospective participants should also be informed of how the information they provide will be disseminated and used—for example, if it will be published in the guideline or made available to other stakeholders on request.

Contact a range of stakeholders by widely advertising opportunities for them to contribute. For example, consider email (including asking relevant organisations to use existing email lists), social media (e.g. Facebook, Twitter, LinkedIn), online newspapers and magazines, advertising through conferences and using the websites of organisations involved in the guideline’s funding and development. It may be worth setting up a system such as an electronic mailing list or online portal for individuals and organisations to register their interest in being involved.

You should ensure that all efforts are made to contact relevant stakeholder groups, including those that may be difficult to reach as not all stakeholders will have the same opportunity to give feedback. You can consider working differently with these stakeholders to help them contribute, or offer alternative ways of engagement to ensure their views are treated equitably. For example, if a stakeholder has limited resources (time, people and money), it might be necessary for you to offer more flexible options in the timing and method of involvement (Department of Prime Minister and Cabinet 2013). Methods to ensure all relevant stakeholders have an opportunity to be involved are discussed further elsewhere (see Equity and Consumer involvement).

Finally, consider your own monetary, time and other resource costs of involving stakeholders in different ways (e.g. people, meeting locations, outreach tools, travel) (see the Project planning module). Try to balance the methods and timing chosen with the funding or existing resources available to the development group (IOM 2011).

5. Consider and respond to stakeholder feedback

Unlike public consultation, it can be relatively straightforward to predict the volume of feedback you will receive from stakeholder engagement. This is because you and the guideline development team have shaped the engagement activity to your needs and targeted a specific audience.

Since they have taken the time to respond to your requests you need to make sure that:

• all stakeholder feedback is documented
• all feedback is taken into consideration
• the participants are acknowledged.

Participants should also have access to documentation on how you handled their comments and any resulting changes. This information could be published with the guideline, provided participants have given permission to do so and were informed of how their feedback will be used before commencing the engagement activity.

You should also consider any new evidence submitted by stakeholders, provided it meets the inclusion criteria used in the evidence review process. If this evidence was published outside the timeframe of the search parameters, the development group may still wish to consider its potential impact and decide if it should be included in the review or elsewhere in your guideline.

NHMRC requirements

Guidelines approved by NHMRC must meet all requirements as outlined in the Procedures and requirements for meeting the NHMRC standard. The following requirements are relevant to involving stakeholders:

F.4 The developer has identified and consulted with key professional organisations (such as specialty colleges) and consumer organisations that will be involved in, or affected by, the implementation of the clinical recommendations of the guideline.

NHMRC Standards

The following Standards apply to the Engaging stakeholders module:

To be relevant and useful for decision making guidelines will:

1.1 Address a health issue of importance.

1.2 Clearly state the purpose of the guideline and the context in which it will be applied.

1.3 Be informed by public consultation.

1.4 Be feasible to implement.

 

To be focused on health and related outcomes guidelines will:

5.1. Be developed around explicitly defined clinical or public health questions.

5.2. Address outcomes that are relevant to the guideline’s expected end users.

5.3 Clearly define the outcomes considered to be important to the person/s who will be affected by the decision, and prioritise these outcomes.

Useful resources

Existing guidance to help identify other developers:

Australian Clinical Practice Guidelines

Centers for Disease Control and Prevention (USA)

NICE Guidance (UK)

Resources on methods of stakeholder involvement:

Department of Health Stakeholder engagement framework

James Lind Alliance Priority Setting Partnerships

NICE Citizens Council

References

Canadian Task Force on Preventive Health Care (2017). CTFPHC Patient Engagement Protocol.

Cluzeau, F., J. A. Wedzicha, et al. (2012). Stakeholder involvement: how to do it right: article 9 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report. Proceedings of the American Thoracic Society 9(5): 269-273.

Cottrell, E., E. Whitlock, et al. (2014). Defining the Benefits of Stakeholder Engagement in Systematic Reviews. Agency for Healthcare Research and Quality, US Department of Health and Human Services.

Department of Health (2015). Stakeholder engagement framework, Australian Government.

Department of Prime Minister and Cabinet (2013). Cabinet Implementation Toolkit, Australian Government.

Green, A. E. and G. A. Aarons (2011). A comparison of policy and direct practice stakeholder perceptions of factors affecting evidence-based practice implementation using concept mapping. Implementation Science 6: 104.

Grol, R., J. Dalhuijsen, et al. (1998). Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. British Medical Journal 317(7162): 858-861.

Institute of Medicine (2011). Clinical practice guidelines we can trust. Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, National Academies of Science.

Jones, C. R. (2011). Using the IRPA Guiding Principles on Stakeholder Engagement: putting theory into practice. Health Physics 101(5): 601-605.

National Institute for Health and Care Excellence (NICE) (2014). Developing NICE guidelines: the manual.

Schmeer, K. (2001). Stakeholder Analysis Guidelines, Policy Toolkit for Strengthening Health Sector Reform, World Health Organisation.

Scottish Intercollegiate Guidelines Network (SIGN) (2015). SIGN 50: a guideline developer’s handbook.

Shin, J. J. (2014). Involving Stakeholders in the Development of Clinical Practice Guidelines. Otolaryngology - Head and Neck Surgery 150(6): 907-909.

Acknowledgments

NHMRC would like to acknowledge and thank Professor Sally Green from Cochrane Australia for her contribution to the development of this module as editor.

Version 5.1. Last updated 22/11/2018.

Suggested citation: NHMRC. Guidelines for Guidelines: Engaging stakeholders. Last published 22/11/2018.