Engaging stakeholders

Engaging stakeholders early and often can be the difference between developing a guideline that sits on the shelf and one that makes a significant impact

Engaging stakeholders

Guidelines have the potential to affect a wide range of people and organisations by influencing behaviour, changing practice and impacting policy, legal or funding frameworks. Stakeholders are individuals or groups who are affected by or can influence a guideline’s development and implementation (IOM 2011), including:

  • government agencies
  • professional societies and the members they represent
  • people affected by the condition (as individuals and collectively in consumer organisations)
  • healthcare and academic centres
  • quality improvement organisations
  • relevant commercial companies
  • other end users.

Stakeholders include ‘expected users’—those who you might expect or intend to apply, implement, or be otherwise directly affected by your guideline recommendations. In this module, stakeholders are classified as anyone with a legitimate interest in your guideline or anyone affected by its recommendations (see Table 1 for examples).

A compelling reason to engage stakeholders is that it can enhance the implementation of your guideline (Grol, Dalhuijsen et al. 1998). First, a guideline developed with input from stakeholders is more likely to be relevant to the needs of its expected users, which improves its implementability. Second, early involvement by organisations that oversee the administration and funding of services that will use your guideline can lead to greater support for its implementation. These benefits are more likely to happen if stakeholders are engaged early in the development process using a consultative approach (Jones 2011).

Table 1. Examples of types of stakeholders in guideline development

Type

Description

Expected users

People intended or expected to apply, implement or be directly affected by the guideline recommendations (e.g. health care providers, policy makers or the public who may use the guideline recommendations to help with decision making).

Government agencies

Government agencies involved in the funding or administration of health care, regulatory and policy activities, such as the Department of Health, state and territory health departments, the Department of Veterans’ Affairs, the Australian Commission for Safety and Quality in Health Care and NHMRC.

Professional bodies

Professional and industrial bodies, such as medical and nursing colleges, specialty societies, industrial groups, scientific networks and public health groups.

Insurers

Private and public insurance companies such as the National Disability Insurance Agency and other payment and reimbursement agencies.

Industry

Industry groups and commercial companies involved in manufacturing, supply or service delivery in health and other relevant sectors.

Condition groups

Groups that focus on a specific disease, health issue or physical state of being (e.g. pregnancy) that support research, represent relevant professionals or advocate for patients or other populations of interest (see also Consumer involvement).

Researchers

Health and medical researchers, institutes and research collaborations.

Service providers

Hospitals, healthcare centres and other relevant service providers including professionals who work within these organisations (e.g. educators).

Advocacy groups

Groups that attempt to influence or lobby legislators and other decision makers on behalf of a particular cause or interest.

You should engage stakeholders at critical points during the development process, such as when the guideline is scoped and when the outcomes are prioritised. While a variety of engagement methods can be used, evidence of their benefits and limitations in guideline development comes from case studies of the experience of developers rather than comparative evaluations. As with all aspects of guideline development, your choice of methods and how you apply them should be balanced with available resources and other constraints (Institute of Medicine 2011) .

The following principles (adapted from Department of Health, 2015) can be used to plan authentic stakeholder engagement:

  • plan in advance for when and how to engage stakeholders
  • involve stakeholders from the start
  • set clear expectations by explaining what their role is and how their contribution will inform the guideline
  • make it easy for stakeholders to be involved, particularly those who may find participation more difficult because of language, culture, age, disability or financial and other barriers
  • be aware of stakeholder objectives, expertise and level of influence
  • acknowledge and respect stakeholder views
  • treat stakeholders equitably.

Some stakeholders, such as consumers, should be involved throughout all stages of guideline development—this advice is discussed in other modules (see Consumer involvement, Guideline Development Group and Public Consultation). The focus of this module is on the general methods of engaging stakeholders throughout guideline development.

What to do

1. Plan for early and repeated stakeholder involvement

Stakeholder perspectives can be considered at different steps in the guideline development process and should be planned in advance.

Some agencies use topics suggested by stakeholders to commission and prioritise new guidelines (NICE 2014; Shin 2014; SIGN 2015). Early involvement might include:

You should provide clear opportunities for the involvement of appropriate stakeholders. These may include reviewing recommendations and derivative products to ensure clear and actionable messaging and planning for implementation (Cluzeau, Wedzicha et al. 2012).

It is important that developers of Australian guidelines involve Aboriginal and Torres Strait Islander Peoples as members of the guideline development group wherever possible, particularly if key outcomes differ in these populations. If this is not possible then you will need to consider and document alternative ways to seek their views as stakeholders. For example, developers of the NHMRC Borderline Personality Disorder guidelines wrote:

Although the Guideline Development Committee membership did not include representation from Aboriginal and Torres Strait Islander peoples, the views of this community were canvassed early in the process through consultation with the Department of Health and Ageing’s Aboriginal and Torres Strait Islander Committee.

Once your guideline has been implemented, stakeholders might contact you with feedback about its usability and relevance. They may also let you know when updates are needed, such as when new evidence becomes available, or when changes in practice or policy occur (SIGN 2015).

While involving stakeholders can boost the relevance and use of your guideline, it can also add to the overall development time and may increase your administrative burden depending on the engagement methods you choose to use (Cluzeau, Wedzicha et al. 2012). This will present a particular challenge if you are developing guidelines with a volunteer workforce, or if you need to develop guidelines quickly, so you should plan accordingly based on your available resources and deadlines (see Organisation and budgeting).

2. Identify local and international stakeholders

Begin this process by conducting a stakeholder analysis. This will help you to determine who should be considered when developing your guideline by systematically gathering and evaluating qualitative information (Schmeer 2001).

First define the expected users of your guideline by thinking about who might use or be affected by its recommendations (Canadian Taskforce on Preventive Health Care, 2017). For example, a guideline on the treatment of muscle strains may be developed for physiotherapists, but other expected users might include general practitioners, primary care nurses and people with muscle strains.

Expected users are only a subset of stakeholders. Other interested parties in guideline development that can also be considered stakeholders include funding bodies and industry groups. Understanding this distinction will help you later when planning how to involve different stakeholders, for instance when deciding which groups should be involved in decisions about your guideline and which others should simply be informed about its development and progress.

It is important to identify not only local stakeholders but also international groups that are developing or publishing advice on similar guideline topics. Searching government websites such as Australian Clinical Practice Guidelines, National Institute for Health and Care Excellence (NICE) Guidance (UK) and Centers for Disease Control and Prevention (CDC) (USA) may help you find guidelines in development or published and it may be helpful to contact their developers for advice on your guideline’s scope and selection of topics.

It is better to be non-selective at this stage to ensure that you identify stakeholders across a wide range of relevant groups, regardless of their interests (see Table 1 for examples of types of stakeholders).

3. Consider levels of involvement for different stakeholders

Once you have identified relevant stakeholders, consider setting up a database and recording stakeholder details such as their:

  • name
  • organisation
  • position
  • contact details
  • location.

It is also useful to record their type—for example, whether they are from the government or private sector, or represent a condition group or professional society.

An important step in determining appropriate levels of involvement is to list all possible stakeholders and consider their degree of relevance to the guideline (Table 2). Relevance in this case refers to how great an impact the guideline may have on them. It can also refer to their capacity to affect, positively or negatively, the development or implementation of the guideline. For example, if you are developing public and environmental health guidelines you should contact and stay in touch with relevant government jurisdictions. You should also align with their advice, since they will likely be ultimately responsible for implementing your guideline.

While not all stakeholders need to be directly engaged, it is important that you consider:

  • who might be interested in your guideline
  • what their views might be
  • what influence they might have on different stages of development.

While the degree of relevance can be a useful measure when determining levels of involvement, it is also important that you consider the interests of all stakeholders and whether these might preclude them from particular engagement activities.

These are important decisions for you to make as the content and implementation of guidelines may change as a result of stakeholder perspectives and whether they are convergent or contrasting (Green and Aarons 2011).

Table 2. Examples of different levels of stakeholder involvement (adapted from Department of Health, 2015).

Level of involvement

Nature of involvement

Example

Collaborate

Partner with stakeholders for the development of mutually agreed solutions and a joint plan of action (two-way or multi-way communication where learning, negotiation and decision making occur on both sides).

This is an option for stakeholders who will be affected by and could influence guideline development and implementation (e.g. organisations that should be involved in implementing the guideline).

Involve

Work directly with stakeholders throughout the process to ensure that issues and concerns are understood and considered (two-way or multi-way communication where learning takes place on both sides).

This is an option for stakeholders who will be affected by the guideline but would not otherwise affect its development (e.g. people who should be targeted to help prioritise outcomes).

Consult

Gain information and feedback from stakeholders to inform decisions made internally (limited two-way communication whereby developers ask questions for the stakeholder to answer).

This is an option for stakeholders who are unlikely to be affected by the guideline but whose activities may influence its development or implementation (e.g. other guideline developers who could help provide evidence tables on related topics).

Inform

Inform or educate stakeholders in one-way communication (there is no invitation to respond).

This is an option for stakeholders who are unlikely to be affected by the guideline or influence its development or implementation (e.g. people who should be sent an email to inform them when public consultation will occur).

 

4. Decide how and when to engage stakeholders

When you are deciding the timing and methods for involving stakeholders you should consider at which stages of development it will be most important to seek their input and plan for distinct engagement activities (Cottrell, Whitlock et al. 2014). You should:

  • decide on an appropriate level of engagement (see Section 3)
  • let the focus and desired outcome of the engagement inform the method of choice
  • avoid tokenistic involvement
  • develop a communications plan which specifies the aim, method and logistics of each engagement activity (see Table 3 for examples of methods).

For all methods it is important that you clearly define which aspects of the guideline are the focus of engagement. Prospective participants should also be informed of how the information they provide will be disseminated and used—for example, if it will be published in the guideline or made available to other stakeholders on request.

Contact a range of stakeholders by widely advertising opportunities for them to contribute. For example, consider email (including asking relevant organisations to use existing email lists), social media (e.g. Facebook, Twitter, LinkedIn), online newspapers and magazines, advertising through conferences and using the websites of organisations involved in the guideline’s funding and development. It may be worth setting up a system such as an electronic mailing list or online portal for individuals and organisations to register their interest in being involved.

You should ensure that all efforts are made to contact relevant stakeholder groups, including those that may be difficult to reach as not all stakeholders will have the same opportunity to give feedback. You can consider working differently with these stakeholders to help them contribute, or offer alternative ways of engagement to ensure their views are treated equitably. For example, if a stakeholder has limited resources (time, people and money), it might be necessary for you to offer more flexible options in the timing and method of involvement (Department of Prime Minister and Cabinet 2013). Methods to ensure all relevant stakeholders have an opportunity to be involved are discussed further elsewhere (see Equity and Consumer involvement).

Finally, consider your own monetary, time and other resource costs of involving stakeholders in different ways (e.g. people, meeting locations, outreach tools, travel) (see Organisation and budgeting). Try to balance the methods and timing chosen with the funding or existing resources available to the development group (IOM 2011).

Table 3. Methods of stakeholder engagement

Method

Description

Examples of use

Stakeholder advisory groups

These structural groups are not involved in developing the recommendations or drafting the guideline but their advice could be sought on a range of issues at any time during development.

These should be formed at the beginning of guideline development with defined terms of reference. E.g. the James Lind Alliance Priority Setting Partnerships enable clinicians, patients and carers to work together to set priority questions for review. They can also be formed during the guideline development process as required to provide input addressing unanticipated topics. The Implementability module also discusses the use of advisory groups to consider implementation issues.

Surveys:

  • Written
  • Telephone
  • Email
  • Face-to-face

These can be used to seek specific advice using structured, straightforward questions. Written surveys are relatively inexpensive and can be used to retrieve feedback from a large population or geographical area, but response rates can be low.

These could be used at any stage of guideline development. Using them early may help to identify individuals and groups who should be consulted in more depth later in the process. E.g. developers of the NHMRC Guidelines for the Management of Overweight and Obesity surveyed consumers and expected users to identify barriers and facilitators to implementation.

In-depth interviews:

  • Telephone
  • Video-conference
  • Face-to-face

 

These can be used to seek detailed information from selected stakeholders, but they can be expensive, difficult to arrange and may need skilled interviewers and analysts. Note also that depending on how these are used, the selection of stakeholders may be divisive if it introduces a hierarchy of importance.

These could be used to seek individual patient preferences or values related to specific treatment options. E.g. developers of the Australian Clinical practice guidelines for antenatal care conducted face-to-face meetings with Aboriginal health workers.

Focus or discussion groups

These can be used to explore specific issues, but they require a skilled facilitator and analyst and it can be difficult to retrieve a variety of different views from participants.

These could be used to explore perceptions within specific groups. E.g. conducting a series of discussion groups in different regions may be useful for the development of national guidelines.

Public meetings and forums

These are opportunities to share information and invite views on a broad range of issues. There might be a risk that discussion is dominated by particular individuals or groups rather than be representative of the whole population. It takes a skilled facilitator to make sure a range of views are presented and to remain focused on particular topics.

These could be used to bring different disciplines together to discuss a topic. E.g. a national public forum was used to develop definitions and principles of care for NHMRC’s National Guidance on Collaborative Maternity Care.

 

Citizen juries

These can be used to explore values and preferences through well informed and structured discussion. They can be useful for sensitive topics where discussion of values and preferences may be divisive.

These could be used to prioritise outcomes and explore different values and preferences on sensitive topics. Depending on the topic, this method may require particular expertise (e.g. health economics) to be effective.

Workshops

These can bring together diverse groups to explore a specific issue or work towards a specific goal. Participants should be carefully recruited depending on the aims of the workshop.

These could be used to bring together implementation experts and expected users to discuss implementation strategies and formulate a plan. E.g. developers of the Australian Guidelines for the Prevention and Control of Infection used workshops to pilot recommendations using the GuideLine Implementability Appraisal (GLIA) tool.

 

5. Consider and respond to stakeholder feedback

Unlike public consultation, it can be relatively straightforward to predict the volume of feedback you will receive from stakeholder engagement. This is because you and the guideline development team have shaped the engagement activity to your needs and targeted a specific audience.

Since they have taken the time to respond to your requests you need to make sure that:

  • all stakeholder feedback is documented
  • all feedback is taken into consideration
  • the participants are acknowledged.

Participants should also have access to documentation on how you handled their comments and any resulting changes. This information could be published with the guideline, provided participants have given permission to do so and were informed of how their feedback will be used before commencing the engagement activity.

You should also consider any new evidence submitted by stakeholders, provided it meets the inclusion criteria used in the evidence review process. If this evidence was published outside the timeframe of the search parameters, the development group may still wish to consider its potential impact and decide if it should be included in the review or elsewhere in your guideline.

NHMRC requirements

Guidelines approved by NHMRC must meet all requirements as outlined in the Procedures and requirements for meeting the NHMRC standard. The following requirements are relevant to involving stakeholders:

F.4 The developer has identified and consulted with key professional organisations (such as specialty colleges) and consumer organisations that will be involved in, or affected by, the implementation of the clinical recommendations of the guideline.

NHMRC Standards

The following Standards apply to the Engaging stakeholders module:

1. To be relevant and useful for decision making guidelines will:

  1.1 Address a health issue of importance.

  1.2 Clearly state the purpose of the guideline and the context in which it will be applied.

  1.3 Be informed by public consultation.

  1.4 Be feasible to implement.

5. To be focused on health and related outcomes guidelines will:

  5.1. Be developed around explicitly defined clinical or public health questions.

  5.2. Address outcomes that are relevant to the guideline’s expected end users.

  5.3 Clearly define the outcomes considered to be important to the person/s who will be affected by the decision, and prioritise these outcomes.

Useful resources

Existing guidance to help identify other developers:

Australian Clinical Practice Guidelines

Centers for Disease Control and Prevention (USA)

NICE Guidance (UK)

Resources on methods of stakeholder involvement:

Department of Health Stakeholder engagement framework

James Lind Alliance Priority Setting Partnerships

NICE Citizens Council

References

Canadian Task Force on Preventive Health Care (2017). CTFPHC Patient Engagement Protocol.

Cluzeau, F., J. A. Wedzicha, et al. (2012). Stakeholder involvement: how to do it right: article 9 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report. Proceedings of the American Thoracic Society 9(5): 269-273.

Cottrell, E., E. Whitlock, et al. (2014). Defining the Benefits of Stakeholder Engagement in Systematic Reviews. Agency for Healthcare Research and Quality, US Department of Health and Human Services.

Department of Health (2015). Stakeholder engagement framework, Australian Government.

Department of Prime Minister and Cabinet (2013). Cabinet Implementation Toolkit, Australian Government.

Green, A. E. and G. A. Aarons (2011). A comparison of policy and direct practice stakeholder perceptions of factors affecting evidence-based practice implementation using concept mapping. Implementation Science 6: 104.

Grol, R., J. Dalhuijsen, et al. (1998). Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. British Medical Journal 317(7162): 858-861.

Institute of Medicine (2011). Clinical practice guidelines we can trust. Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, National Academies of Science.

Jones, C. R. (2011). Using the IRPA Guiding Principles on Stakeholder Engagement: putting theory into practice. Health Physics 101(5): 601-605.

National Institute for Health and Care Excellence (NICE) (2014). Developing NICE guidelines: the manual.

Schmeer, K. (2001). Stakeholder Analysis Guidelines, Policy Toolkit for Strengthening Health Sector Reform, World Health Organisation.

Scottish Intercollegiate Guidelines Network (SIGN) (2015). SIGN 50: a guideline developer’s handbook.

Shin, J. J. (2014). Involving Stakeholders in the Development of Clinical Practice Guidelines. Otolaryngology - Head and Neck Surgery 150(6): 907-909.

Acknowledgments

NHMRC would like to acknowledge and thank Professor Sally Green from Cochrane Australia for her contribution to the development of this module as editor.

 

Version 5.1. Last updated 22/11/2018.

Suggested citation: NHMRC. Guidelines for Guidelines: Engaging stakeholders. https://nhmrc.gov.au/guidelinesforguidelines/plan/engaging-stakeholders. Last published 22/11/2018.

ISBN: 978-1-86496-024-2