Types of studies

Randomised controlled trials (RCTs)

Types of participants

Qualified health care professionals responsible for patient care or health professionals in post‐graduate training (e.g. resident physicians). Studies involving only undergraduate students will be excluded.

Types of interventions

Any formal or informal local consensus process (alone or as a component of a multifaceted interventions) aimed at promoting implementation of the guideline. Local consensus processes are defined as the inclusion of targeted healthcare providers in discussion to ensure that they agree on one or more elements of the guideline, including importance of the clinical problem and the approach to managing the problem.

We consider multifaceted interventions as those which include two or more interventions and local consensus process is one of the interventions. The other interventions could include audit and feedback, reminders, marketing and etc.

Types of outcome measures

Any objective measure of:
(1) Health care professional practice in a healthcare setting.
(2) Health care (patient oriented) outcomes.
Where either (1) or (2) has been satisfied then outcome (3) will be reported if available:
(3) Professional satisfaction and any other relevant outcome measures.
Where either (1) or (2) has been satisfied then outcome (4) may also be considered:
(4) Resource use and cost of the local consensus process and the comparison interventions.
Studies that measure knowledge or performance in a test situation only will be excluded.

Screening

Two review authors (MN and EP) will independently read the titles and/or abstracts resulting from the search process and eliminate any obviously irrelevant studies. The remaining studies classified as clearly relevant or unclear will be retrieved in full text. The same two authors will independently assess these for inclusion. Difference in opinion that cannot be resolved by consensus between the two authors will be discussed with the third author (ZF).

Quality assessment

Assessment of methodological quality will be based on EPOC guidelines. The quality of all eligible studies will be assessed by two independent authors using criteria described in the EPOC module, i.e. data regarding design, participants, interventions and outcomes with assessment of methodological quality based on seven criteria for RCTs: concealment of allocation, blinded or objective assessment of primary outcome(s), and completeness of follow up of professionals/patients and no important concerns in relation to baseline measures, reliable primary outcomes or protection against contamination. After assessment the included studies will be grouped accordingly.
(A) Low risk of bias (plausible bias unlikely to seriously alter the results): if the three first criteria are met and there are no important concerns related to the last three criteria.
(B) Moderate risk of bias (plausible bias that raises some doubt about the results): if one or two criteria are scored as not clear or not done
(C) High risk of bias (plausible bias that seriously weakens confidence in the results): If more than two criteria are scored as not clear or not done

For cluster randomisation trials protection against contamination will be rated as done. Any discrepancies in quality ratings will be resolved by discussion and involvement of the third author, if needed.

We have defined multifaceted interventions as including two or more interventions (e.g. local consensus processes and reminders). For multifaceted interventions that include local consensus processes two authors will independently categorise the contribution of local consensus processes as a component of the intervention as a major, moderate or minor component.

The complexity of the targeted behaviour will be categorised independently by two authors as high, moderate or low. The categories will depend upon the number of behaviours required, the extent to which complex judgements or skills are necessary, and whether other factors such as organisational change are required for the behaviour to be improved and also depend on whether there is need for change only by the individual/professional (one person) or communication change or change in systems. If an intervention is targeted at relatively simple behaviours, but there are a number of different behaviours, e.g. compliance with multiple recommendations for prevention, the complexity will be assessed as moderate.

The seriousness of outcome will also be categorised independently by two authors as high, moderate or low. Acute problems with serious consequences will be considered as high. Primary prevention will be considered moderate. Numbers of unspecified tests or prescriptions will be considered low.
Baseline compliance with the targeted behaviours will be treated as a continuous variable ranging from zero to 100%, based on the pre‐intervention level of compliance as a mean for both or all groups before the intervention.

We will use the following definitions for interventions other than local consensus processes: educational materials: distribution of published or printed recommendations for clinical care, including clinical practice guidelines, audio‐visual materials and electronic publications.

Patient mediated interventions: any intervention aimed at changing the performance of healthcare providers indirectly by providing information, prompts, or support to the patient.

Reminders: any intervention, manual or computerised, that prompts the healthcare provider to perform some action.

Tailoring: use of personal interviewing, group discussion ('focus groups'), or a survey of targeted providers to identify barriers to change and subsequent design of an intervention that addresses identified barriers.

Data extraction
Data will be extracted by two authors (MN and ZF) independently. The EPOC data collection checklist will be used to include information on study design, type of intervention, presence of controls, type of targeted behaviour, participants, setting, methods (unit of allocation, unit of analysis, study power, methodological quality, consumer involvement), outcomes and results. Discrepancies between authors will be resolved through discussion; decisions that cannot be easily resolved will be referred to the contact editor for the review. Where necessary, additional primary data will be obtained through collaboration with the original trial authors.

Data analysis

Comparisons that randomise or allocate clusters (professionals or healthcare organisations) but do not account for clustering during analysis have 'potential unit of analysis errors' resulting in artificially low P values and overly narrow confidence intervals. We will re‐analyse studies with potential unit of analysis errors where possible. If this is not possible we will report only the point estimate.
We will only include studies of moderate or high quality in the primary analyses, and studies that report baseline data. All outcomes will be expressed as compliance with desired practice. Professional and patient outcomes will be analysed separately.

When several outcomes are reported in one trial we will only extract results from the primary outcome. If there are more than one primary outcome or if the primary outcome is not specified, we will calculate the median effect size for the outcomes reported in the trial.

We will consider the following potential sources of heterogeneity to explain variation in the results of the included studies:

• the type of intervention (local consensus processes alone, or multifaceted interventions that included local consensus processes);

• the contribution of local consensus processes as a component of the intervention for multifaceted interventions;

• setting of care (primary versus hospital);

• complexity of the targeted behaviour;

• seriousness of the outcome;

• baseline compliance;

• study quality (high or moderate protection against bias);

• the focus of the consensus (e.g. adoption or modification of existing guidelines, or development of new guidelines);

• the consensus process that was used.

Potential heterogeneity will be explored visually by preparing tables, bubble plots (where the size of the bubble correspond to the number of healthcare professionals who participated) and box plots (displaying medians, interquartile ranges, and ranges) to explore the size of the observed effects in relationship to each of these variables.

Each comparison will be characterised relative to the other variables in the tables, looking at one potential explanatory variable at a time. We will look for patterns in the distribution of the comparisons, hypothesising that larger effects will be associated with multifaceted interventions, more intensive consensus processes, less complexity of the targeted behaviour, more serious outcomes, lower baseline compliance, and lower study quality.

The visual analyses will be supplemented with multivariate statistical analyses, if there are sufficient data. Weighted meta‐regression will be used to examine how the size of the effect was related to the 10 potential explanatory variables listed above; weighted according to the number of healthcare professionals. These analyses will be conducted using generalized linear modelling in SAS (version 8.2. SAS Institute Inc., Cary, NC, USA). Two main analyses will be conducted for comparison 1. Any intervention in which educational meetings is a component compared to no intervention, with or without printed educational materials: one using the adjusted risk ratio and one using the adjusted risk difference as the dependent variable.

In order to minimize the risk of spurious estimates of effect from the meta‐regression due to a high number of independent variables compared to the number of studies in the analysis, the meta‐regression will be performed in a stepwise manner with three steps:

(1) Each of the potential explanatory variables will be analysed as the only independent variable in a meta‐regression in order to assess an unadjusted baseline effect ‐ variables with a P‐value > 0.3 will be excluded as explanatory variables in step 3.
(2) We will examine interactions between the following variables and the type of intervention: the intensity of educational meetings, and interactive versus didactic educational meetings. Interaction‐terms with a P‐value > 0.3 will be excluded from further analysis.
(3) Explanatory variables from 1 (P‐value <= 0.3) and interactions from 2 will be combined into the final meta‐regression‐model.

In cases of potentially important baseline differences in compliance between intervention and control groups in trials, the primary analyses will be based on adjusted estimates of effect, where we will adjust for baseline differences in compliance. For dichotomous outcomes we will calculate the adjusted risk difference and risk ratio as follows:

Adjusted risk difference (RD) = the difference in adherence after the intervention minus the difference before the intervention. A positive risk difference indicates that adherence improved more in the educational intervention group than in the control group, e.g. an adjusted risk difference of 0.09 indicates an absolute improvement in care (improvement in adherence) of 9%.

Adjusted risk ratio (RR) = the ratio of the relative probability of adherence after the intervention over the relative probability before the intervention. A risk ratio greater than one indicates that adherence improved more in the educational intervention group than in the control group, e.g. an adjusted risk ratio of 1.8 indicates a relative improvement in care (improvement in adherence) of 80%.

For continuous outcomes we will calculate the post mean difference, adjusted mean difference and the percent change relative to the control mean after the intervention.

For studies with no unit of analysis error and with moderate or high quality and reported baseline data, we will record the adjusted odds ratio (or other measure of effect) and P‐values or confidence intervals reported by the authors. If possible, we will compare these results with our analyses to assess the robustness of our analyses.