Evidence-based guidelines for better health: can their role be sustained?

This project matters because it recognizes a threat to the continued production or modifications (adaptation) of guidelines, which promote best health care practices; and, it offers a framework for solving this threat. Over-emphasis of unnecessarily stringent standards for guideline development methods beyond what is reasonable and practical threatens the sustainability of affordable evidence-based guideline production by adding costs and production delays. A framework is proposed to allow for affordable health care guidelines that are ‘good enough’ to be trustworthy.

Fig. 1. The Efficiency-Validity Methodological Continuum for Sustainable Guideline Development.

Recommendations contained in guidelines for health care (and policy) are most trustworthy when based on research evidence (evidence-based guidelines – EBGs). But EBGs must be carefully designed using rigorous methods to maintain their value as trusted resources. Some steps in EBG development methods (quality standards) can be impractical and expensive to apply, with only slight gains in EBG trustworthiness.

This project proposes a framework for more efficient development of affordable EBGs that are ‘good enough’. EBGs summarize results from health research to make recommendations for best practice. Their recommendations are used by doctors and other healthcare professionals to advise patients about treatment and lifestyle choices. Regulators and policy makers use EBGs to inform decisions about approving new products, including drugs.

EBGs have come under attack as variable in their quality, and therefore their trustworthiness, requiring a tightening of methodological standards in their development.  In 2011 the US Institute of Medicine (IOM) released ‘methodological standards’ for EBG development, many of which are costly and impractical to apply. By being evidence-based (based on research studies) EBGs already exceed methodological standards of many current guidelines. We asked, “How much higher do standards have to be before a guideline recommendation can be trusted?” Or, when is an EBG ‘good enough’? Adhering to all of the IOM standards for every EBG threatens the future of the EBG movement in two ways – higher production costs for which sponsors are less willing to pay, and delays in development that cause doctors to look elsewhere for guidance.

The proposed framework (Fig. 1.) for developing ‘good enough’ guidelines recognizes that some methodological standards are more important than others, that those who would be affected by a guideline’s recommendations (stakeholders) should be represented in EBG development decisions, and that negotiations among these stakeholders, guided by methodological experts, should replace sole reliance on such experts in decision making about what methodological compromises are reasonable and safe. Although the framework advocates for practical methodological shortcuts, it also improves development methods in three important ways compared to current processes: greater involvement of those to be affected by a guideline; educational value for stakeholders regarding methodological standards; and greater transparency and accountability in decisions about methodological compromises and the risks taken.

The framework serves as a tool for stakeholder negotiations (Fig. 1. – the Efficiency-Validity Methodological Continuum). It acknowledges risks to validity (trustworthiness) when some methodological short-cuts are taken to gain efficiencies in production. It empowers stakeholders (guideline investors, patient representatives and healthcare workers) to negotiate reasonable tradeoffs between threats to the trustworthiness of a guideline, and the efficiencies gained to achieve a ‘good enough’ EBG.

The framework is represented as a ‘tradeoff continuum’ with negotiated zones of ‘acceptability’ and ‘preference’ for compromise. Positioned between these zones are extremes to be avoided, where extra expense is not worth each extra degree of trustworthiness (validity) gained, and where the risk to trustworthiness is not worth the extra efficiencies gained. Zones of compromise will differ depending on the threat to health of the underlying medical condition, stakeholder values, and risks to health from certain methodological shortcuts.

The framework applies best to local guideline development or adaptation processes.

George P. Browman
School of Population and Public Health, University of British Columbia
Department of Clinical Epidemiology & Biostatistics, McMaster University



When is good, good enough? Methodological pragmatism for sustainable guideline development.
Browman GP, Somerfield MR, Lyman GH, Brouwers MC.
Implement Sci. 2015 Mar 6


Leave a Reply