Guidelines for Guidelines

A transparent guideline should give users confidence that it is based on the best available evidence, is largely free from bias and is clear about the purpose of its recommendations (IOM 2011).

If your guideline does not follow a transparent development process then users may question its credibility, its relevance and its applicability to them. It may also result in you wasting valuable time, money and resources if you have to revisit, unpick and defend key steps in the guideline development process. Importantly, lobby groups and aggrieved parties may use gaps in transparent reporting as an opportunity to undermine public trust in your guideline.

Unfortunately Australian guidelines often lack the level of transparent reporting that users require to place their trust in their recommendations. Of a sample of more than 1000 Australian guidelines published between 2004 and 2013, 66% did not include a description of the development process (NHMRC 2014).

You are not expected to make every minute detail of your guideline development process accessible to the public, however it is important to know what information should be documented, what should be made public, and to what level of detail.

What to do

1. Consider where transparency fits into the guideline development process

Guideline users need to understand the processes and procedures used to develop your guideline. As a guideline developer you should be aware that ensuring transparency is not restricted to one stage of guideline development: it is an ongoing process throughout the development process.

Things that must be publicly available include:

• the governance processes you employed, including:
  • the terms of reference of the guideline development group,
  • details on how the group was recruited, and
  • how conflicts of interest were managed
• a summary of the guideline development methods you used
• disclosure of all sources of funding
• submissions from public consultations, and how these submissions were considered or incorporated into the guideline
• submissions made by independent reviewers, and how these submissions were considered or incorporated into the guideline.

In assessing whether a guideline can be trusted, users will often look for the following assurances:

• the guideline funders will not profit from its recommendations
• the guideline is based on the best available, rigorously-sourced and graded evidence
• the guideline is hosted on a trustworthy website
• the guideline has been drafted by people that will be using and implementing the guideline, researchers in the relevant area of interest, and people likely to be affected by its recommendations
• the guideline development group members do not have a conflict of interest
• there is a process to manage possible conflicts of interest
• bias has been managed during the systematic review process
• the guideline development group has documented how it arrived at the recommendations
• the guideline has been independently reviewed.

2. Understand what guideline users will be looking for

Your guideline will attract a potentially wide range of users (see the Engaging stakeholders module) who should all be able to find their way, clearly and logically, from the recommendations back along the evidence chain to the original questions and scope of the guideline, and understand how and why key decisions have been made.

The use of GRADE and Evidence to Decision frameworks are important parts of this process. These both set up a systematic discussion about the evidence in order to arrive at a recommendation.

3. Be clear on what you will be documenting

You will also need to prepare and publish information relevant to the administrative and technical process of the guideline. These are often called Administrative Reports and Technical Reports and can be published as part of the guideline or as separate documents.

Administrative Reports typically include information on:

• all sources of funding used for the guideline, with any relevant caveats
• guideline development group membership:
  • their position, credentials, affiliations and role
  • the expertise they bring
  • how they were recruited
• staff employed to work on the guideline
• project governance, including  declaration and management of interests
• how public consultation was planned and managed.

Technical Reports will provide information on:

• the steps in the systematic review process, including development of the clinical questions
• the evidence search strategy for each clinical question, including search strings for the different databases and cut-off dates
• the screening of studies, synthesis of the evidence, evidence profiles and summary of findings tables,
• grading of the evidence, and formulation of recommendations
• information about any stakeholder and consumer consultation processes and independent expert reviews.

For an example of the level of detail displayed for working groups, conflict of interest management and the evidence review process see the Clinical Guidelines for Stroke Management, published on MAGICapp.

For an example of the level of detail documented on formulating the recommendations see the Clinical guideline for the diagnosis and management of work-related mental health conditions in general practice. 

4. Select the best available tools 

Many tools are available that can help improve the transparency of information documented in guidelines. 

The Resources section below provides links to the key tools that can be used at each stage of the guideline development process to improve the transparency of your guideline. Many of them are checklists that will help you to frame the procedures of your guideline development project, and help to track your progress.

5. Commission independent review

Independent review of draft guidelines is a process step in guideline development of NHMRC, NICE (National Institute for Health and Care Excellence) and SIGN (Scottish Intercollegiate Guidelines Network) guidelines. Independent review by peers, experts and consumers allows the inclusion of views from outside the guideline development group. 

Principles of independent review include transparency of the review process; review by someone who has relevant expertise and experience; confidentiality; the provision of objective and impartial advice; and the review should be undertaken with reference to international standards and quality indicators. Independent reviewers will be subject to the same conflict of interest policy as any other participant in the guideline development process. 

A team of individual reviewers may undertake a review of different aspects of the document, depending on their area of expertise, if appropriate. The reviewers must document and provide a record of their comments. As with feedback received from the public consultation process, reviewer comments must be responded to, with records kept of any changes made, and if not, the reasons for not incorporating the comment. 

The Independent review module covers this in more detail.

6. Make your documents accessible

Your guideline and associated administrative and technical reports need to be accessible and easy to find. Contact details should be provided so that guideline users can request additional information if required.

Publicly available information, contained in appendices or associated stand-alone reports, should include:

• governance processes
• a ‘living’ project plan — the rationale and scope for the guideline; the intended end users and target populations; project timelines; potential funding procurement activities; a risk assessment and management plan; reporting requirements; a budget
• consultation plan — including timelines, types of consultation (that is, face to face or online) and locations of consultation meetings, groups consulted during the process and how they were recruited
• media management strategy (if required)
• scoping document
• evidence review report including the search strategy, assessment, synthesis of the evidence, risk of bias considerations, grading
• outline of the process for drafting recommendations
• comments from the public consultation process — a record must be kept of all comments, whether they were incorporated into the guideline, and if not the reasons for not incorporating them
• report by the independent reviewer, and the guideline development group’s response to the comments.

7. Publish your guideline on a trustworthy website

Websites that publish guidelines and other health information should provide evidence of their credentials to assure users that they are legitimate and trustworthy. This may include: 

• showing evidence of partnership arrangements with recognised information providers, such as the Australian Government’s Health Direct information partner scheme
• providing active and prominent links to parent organisations
• providing evidence of endorsement or approval by relevant and reputable external organisations such as professional colleges, Standards Australia and NHMRC
• discouraging links to your guideline page from sites that sell services, products or advice
• explicitly identifying any paid advertising on your website.
• ensuring your ‘about us’ and ‘contact us’ links are active and current, and update or rescind any information that is no longer current and relevant.

Resources and checklists

• AGREE II: a tool for assessing the quality and reporting of guidelines
• Australian Privacy Principles quick reference guide: covers the collection, handling, use and disclosure of personal information for Australian organisations with responsibilities under the Privacy Act 1988
• G-I-N-McMaster University Guideline Development Checklists: includes a seven-item checklist for the guideline group membership and a ten-item checklist for establishing guideline group processes. The checklists cover such issues as roles and responsibilities, documentation of the group selection process; induction and training; establishing methods for conflict resolution; and the keeping of records for all meetings.
• GUIDE-M: a mind map that outlines the steps in the process of a systematic review and emphasises the need for guideline validity and reproducibility
• NHMRC requirements
• PRECEPT framework: for evaluating and grading evidence in the field of public health.
• PRISMA: a checklist for putting together a standardised record of studies during your evidence identification phase, and will help you record a number of details about the study involved
• RIGHT Checklist: assists developers to summarise the key evidence from guidelines including health issue covered, aims and objectives of the guideline, evidence used, how recommendations were developed.
• SIGN50 Methodology Checklists: six checklists for systematic reviews, randomised controlled trials, cohort studies, case controlled studies, diagnostic studies and economic studies
• The NEATS instrument is a focused tool that provides a concise evaluation of a guideline's adherence to the IOM standards for trustworthy guidelines
• TIDiER Checklist:  a tool for reporting, describing and reproducing interventions.

Note: a full listing of risk of bias tools is available in the Assessing risk of bias module.

References

Institute of Medicine (2011). Clinical Practice Guidelines We Can Trust. Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, National Academies Press.

National Health and Medical Research Council (NHMRC) (2014). Australian Clinical Practice Guidelines 2014 Annual Report.