Original article
Attributes of clinical recommendations that influence change in practice following audit and feedback

https://doi.org/10.1016/S0895-4356(02)00403-1Get rights and content

Abstract

The object of this study was to determine which attributes of clinical practice recommendations influence changes in clinical practice following audit and feedback. This was an observational study using multilevel modeling to examine the relationship between attributes of clinical practice recommendations and compliance with the recommendations before and after audit and feedback. Sixteen hospital gynecology units in Scotland participated in a national audit project. Clinical practice recommendations covering selected gynecological topics were developed and data collected to assess baseline (preintervention) compliance. Summaries of performance were fed back to consultant gynecologists in each hospital and follow-up (postintervention) data were collected. Trained audit assistants used standardized forms to abstract data from case notes. Compliance data were available at baseline and follow-up for a total of 42 clinical practice recommendations. Altogether, 4,664 case notes contributed to baseline data and 4,382 to follow-up data. Thirteen attributes describing clinical practice recommendations were developed, based upon previous work, and pretested. A panel of seven consultant gynecologists rated the extent to which each of the 42 recommendations possessed each of the 13 attributes. The main outcome measures were the association of each attribute with compliance and with changes in clinical practice. Recommendations compatible with clinician values and not requiring changes to fixed routines were independently associated with greater compliance at baseline and follow-up. However, recommendations incompatible with clinician values were independently associated with greater change in practice following audit and feedback. Attributes of recommendations may influence the effectiveness of audit and feedback in secondary care. Recommendations seen as incompatible with clinician values are associated with lower compliance but greater behavioral change following audit and feedback.

Introduction

The implementation of valid clinical guidelines can improve the quality of health care [1]. Passive dissemination of a guideline is unlikely to lead to changes in clinical practice [2]. Combining more active interventions, such as reminders or interactive education, with guideline dissemination and implementation is more likely to change professional and organizational practice.

Various factors, or effect modifiers, can influence the effectiveness of such interventions [3]. Until recently, most research has focused on characteristics of clinicians or health care organizations, such as local attitudes or preparedness to change. However, the characteristics of clinical practice recommendations themselves may also influence their rate of adoption [4].

Grilli and Lomas first assessed the association between such characteristics and compliance with clinical guideline recommendations [5]. They reviewed published studies reporting compliance rates with 143 different recommendations developed or endorsed by official organizations. Compliance was higher for recommendations displaying “trialability” (which could be tried out temporarily and discarded if found wanting) and lower for complex recommendations. The “observability” of recommendations (how readily their benefits could be seen to be achieved) had no impact.

Grol et al. [6] assessed the extent to which Dutch general practitioners' compliance with 47 guideline recommendations was influenced by 12 characteristics (or attributes) of the recommendations. The guidelines were disseminated via journals and continuing medical education programs. Compliance was lower if recommendations were vaguely worded, incompatible with clinician norms and values, and disruptive to routine practice.

This previous work focused on the effects of various attributes on compliance with recommendations (i.e., a one-off measure of performance). However, clinical guidelines are produced to promote change in behavior (i.e., hereafter referring to a decrease or increase in compliance), which may be influenced by attributes different from those associated with compliance. We investigated whether various attributes of clinical practice recommendations influenced both compliance and change in clinical behavior among specialists participating in a national audit program.

Section snippets

Methods

Table 1 summarizes the context and main steps of this study, now described in detail.

Panel rating of GAPS recommendations

The number of clinical recommendations displaying each attribute varied widely (Table 2). For example, 40 (95%) of the 42 recommendations were judged to precisely describe recommended clinical practice, and 41 (98%) as addressing a common clinical issue. None were judged as requiring new knowledge or skills, and only 3 (7%) each as trialable or as complex.

The five attributes (precisely described, addresses common issue, requires new knowledge or skills, complex and trialable) displayed by over

Principal findings

Certain attributes of clinical practice recommendations were associated with variations in compliance and behavior change before and after an audit and feedback program in secondary care. Consistent with previous work [6], recommendations compatible with clinician norms and values and not requiring changed routines were independently associated with higher compliance. However, clinical practice recommendations are intended to change behavior, and a different picture emerged when changes in

Contributors

R.F. participated in designing the study, running the consensus panel, and data analysis. G.M. analyzed the data. J.G. suggested the original idea for the study, and participated in its design and analysis. G.P. participated in design of the study, operation of the consensus panel, and the original audit data collection. M.C. participated in design and analysis. R.G. participated in design of the study. All contributors participated in the writing of the study. R.F. is guarantor for the study.

Acknowledgements

The Health Services Research Unit is funded by the Chief Scientist Office (CSO), and the Scottish Programme for Clinical Effectiveness in Reproductive Health is funded by the Clinical Resource and Audit Group, both of the Scottish Executive Health Department. Robbie Foy was funded by a Medical Research Council/CSO training fellowship in health services research. However. the views expressed reflect those of the authors and not the funding bodies. We are grateful to the consultant gynecologists

References (20)

  • S.H Woolf et al.

    Potential benefits, limitations, and harms of clinical guidelines

    BMJ

    (1999)
  • L.A Bero et al.

    Closing the gap between research and practicean overview of systematic reviews of interventions to promote the implementation of research findings

    BMJ

    (1998)
  • J Lomas

    Teaching old (and not so old) docs new trickseffective ways to implement research findings

  • E.M Rogers

    Diffusion of Innovations

    (1995)
  • R Grilli et al.

    Evaluating the messagethe relationship between compliance rate and the subject of a practice guideline

    Med Care

    (1994)
  • R Grol et al.

    Attributes of clinical guidelines that influence use of guidelines in general practiceobservational study

    BMJ

    (1998)
  • M.A Thomson O'Brien et al.

    Audit and feedback versus alternative strategieseffects on professional practice and health care outcomes. The Cochrane Library

    (2001)
  • G.C Penney et al.

    A multi-centre, criterion-based audit of the management of induced abortion in Scotland

    BMJ

    (1994)
  • G.C Penney et al.

    Laparoscopic sterilisationopinion and practice among gynaecologists in Scotland

    Br J Obstet Gynaecol

    (1997)
  • G.C Penney et al.

    Endometrial assessment proceduresan audit of current practice in Scotland

    Hum Reprod

    (1997)
There are more references available in the full text version of this article.

Cited by (138)

  • Systematic review of current guideline appraisals performed with the Appraisal of Guidelines for Research & Evaluation II instrument—a third of AGREE II users apply a cut-off for guideline quality

    2018, Journal of Clinical Epidemiology
    Citation Excerpt :

    No tool used a cut-off to distinguish between high- and low-quality guidelines. Only one tool applied a cut-off to determine whether recommendations fulfill specific requirements, but the authors did not present a rationale for the cut-off applied [22]. A cut-off would be useful to issue a transparent and standardized recommendation for a guideline.

View all citing articles on Scopus
View full text