Intended for healthcare professionals

General Practice

Diabetes care in general practice: meta-analysis of randomised control trialsCommentary: Meta-analysis is a blunt and potentially misleading instrument for analysing models of service delivery

BMJ 1998; 317 doi: https://doi.org/10.1136/bmj.317.7155.390 (Published 08 August 1998) Cite this as: BMJ 1998;317:390

Abstract

Objective : To assess the effectiveness of care in general practice for people with diabetes.

Design : Meta-analysis of randomised trials comparing general practice and shared care with follow up in hospital outpatient clinic.

Identification : Trials were identified from searches of eight bibliographic and research databases.

Results : Five trials identified included 1058 people with diabetes, overall mean age 58.4 years, receiving hospital outpatient follow up for their diabetes. Results were heterogeneous between trials. In shared care schemes featuring more intensive support through a computerised prompting system for general practitioners and patients, there was no difference in mortality between care in hospital and care in general practice (odds ratio 1.06, 95% confidence interval 0.53 to 2.11); glycated haemoglobin tended to be lower in primary care (weighted difference in means of 0.28%, 0.59% to 0.03%); and losses to follow up were significantly lower in primary care (odds ratio 0.37, 0.22 to 0.61). However, schemes with less well developed support for family doctors were associated with adverse outcomes for patients.

Conclusions : Unstructured care in the community is associated with poorer follow up, worse glycaemic control, and greater mortality than in hospital care. Computerised central recall, with prompting for patients and their family doctors, can achieve standards of care as good as or better than hospital outpatient care, at least in the short term. The evidence supports provision of regular prompted recall and review of selected people with diabetes by willing general practitioners. This can be achieved if suitable organisation is in place.

Diabetes care in general practice: meta-analysis of randomised control trials

  1. Simon Griffin, Wellcome training fellow (SJG49{at}medschl.cam.ac.uk)
  1. Primary Medical Care Group, University of Southampton, Aldermoor Health Centre, Southampton SO16 5ST
  2. University College London Medical School/Royal Free Hospital School of Medicine, Whittington Hospital, London N19 5NF
  1. Correspondence to: Dr Simon Griffin, General Practice and Primary Care Research Unit, Department of Community Medicine, Institute of Public Health, Cambridge CB2 2SR
  • Accepted 6 April 1998

Abstract

Objective : To assess the effectiveness of care in general practice for people with diabetes.

Design : Meta-analysis of randomised trials comparing general practice and shared care with follow up in hospital outpatient clinic.

Identification : Trials were identified from searches of eight bibliographic and research databases.

Results : Five trials identified included 1058 people with diabetes, overall mean age 58.4 years, receiving hospital outpatient follow up for their diabetes. Results were heterogeneous between trials. In shared care schemes featuring more intensive support through a computerised prompting system for general practitioners and patients, there was no difference in mortality between care in hospital and care in general practice (odds ratio 1.06, 95% confidence interval 0.53 to 2.11); glycated haemoglobin tended to be lower in primary care (weighted difference in means of 0.28%, 0.59% to 0.03%); and losses to follow up were significantly lower in primary care (odds ratio 0.37, 0.22 to 0.61). However, schemes with less well developed support for family doctors were associated with adverse outcomes for patients.

Conclusions : Unstructured care in the community is associated with poorer follow up, worse glycaemic control, and greater mortality than in hospital care. Computerised central recall, with prompting for patients and their family doctors, can achieve standards of care as good as or better than hospital outpatient care, at least in the short term. The evidence supports provision of regular prompted recall and review of selected people with diabetes by willing general practitioners. This can be achieved if suitable organisation is in place.

Key messages

  • Responsibility for the routine recall and review of people with diabetes has shifted to primary care

  • Central computerised, prompted recall and review for people with diabetes in primary care can achieve outcomes as good or better than follow up in hospital

  • The transfer of responsibility for diabetes care to general practitioners without adequate support is associated with adverse outcomes for patients

  • The cost effectiveness of general practice diabetes care needs longer term evaluation

Introduction

The important and necessary involvement of general practice in diabetes care is well recognised.15 Since 1970, increasing numbers of family doctors in the United Kingdom have assumed responsibility for the routine review of their patients with diabetes, for a variety of underlying reasons, although it is sometimes difficult to assess whether care has been shared or simply shifted. 3 6

Evaluation of diabetes care in the community has produced conflicting results. Satisfactory follow up in primary care has been far from universal and cannot be guaranteed,79 but in certain circumstances general practitioners have achieved follow up and metabolic control at least as good as their hospital colleagues.1012

This study aimed to identify and evaluate all published randomised trials of hospital versus general practice care for people with diabetes, to compare the effectiveness of general practice and hospital care through the use of meta-analysis of the identified trials, and to explore variations in the findings of the individual trials.

Methods

Identification of relevant trials

Eight bibliographic and research databases were searched. The medical subject heading “diabetes” was combined with each of the terms in the box to identify all studies, in any language, indexed on Embase, CRIB, or Dissertation Abstracts. The search terms were then combined with the Cochrane Collaboration strategy for identifying randomised trials13 on Medline, National Research Register, CINAHL, PsychLit, and Healthstar. Finally, bibliographies of trials identified by computerised search were hand searched for further references to trials. Studies were included in which people with diabetes (insulin dependent or non-insulin dependent) were randomly allocated to hospital or to general practice or shared care for routine review and surveillance for complications, regardless of the quality of concealment of allocation or choice of outcome measures.

Data extraction

Descriptive data about each trial were extracted from the published reports; original authors were contacted for clarification when details about randomisation were not reported. Values for the following outcomes, where available, were extracted for hospital and general practice groups: means (and standard deviations) for glycated haemoglobin, final systolic and diastolic blood pressure, patient and health service costs, the number of diabetes reviews and glycated haemoglobin estimations per patient per year, and the numbers of patients dying, admitted to hospital, referred to dietitians and chiropodists, and lost to follow up. Losses to follow up were calculated as the mean number not attending each of the three stipulated interim appointments between annual reviews in one study14 and the number with no record of an annual review in the rest.

Search terms

View this table:
TABLE 1

Characteristics of trials included in meta-analysis of diabetes care in general practice

View this table:

Statistical methods

Peto odds ratios, weighted difference in means, and χ2 tests of intertrial heterogeneity were calculated by using the fixed effects model of the Cochrane review manager software.15 The denominator used to calculate effect sizes for mortality, follow up, and hospital admissions was the number of subjects randomised. The denominator for the remaining effect sizes was the number of subjects in whom that outcome had been assessed.

Sensitivity analysis

To determine if the findings were robust to different analyses, a random effects model was used, and the numbers of subjects randomised and the numbers in whom the outcome was assessed were substituted into the denominator for each of the comparisons. A stratified analysis was performed according to whether the model of shared care was basic or computer assisted (with a central computer to prompt both family doctors and patients to undertake protocol driven diabetes reviews), after the taxonomy of Hickman et al.16 In addition, trials were stratified by publication year, the proportion of local practices taking part, and whether patients treated with insulin were included.

Results

The combined searches identified over 1200 studies, but only five met the inclusion criteria (table 1). All five trials used satisfactory randomisation of individual subjects, but they were of short duration, only one lasting more than 2 years.17 In aggregate, 1058 people seen in hospital diabetes clinics were eligible and agreeable to randomisation to continuing hospital outpatient review or follow up in the community, either by their family doctor alone or as part of a shared care scheme. The organisation of care for the hospital outpatient group was not clearly defined, although the descriptions seem broadly similar. All the general practitioners were provided with educational sessions or protocols before the trials. However, the support for care in general practice changed over time. Two studies published in the 1980s evaluated basic general practice care, 17 18 two more recent studies included computer prompting systems, 19 20 and one recent trial by Hoskins et al compared both basic and prompted general practice care with hospital care and is therefore included in the table as two separate studies.14

Fig 1
Fig 1

Weighted difference in mean percentage of glycated haemoglobin between general practice and hospital care. Bars indicate 95% confidence intervals

Fig 2
Fig 2

Odds ratios of mortality in general practice and hospital care. Bars indicate 95% confidence intervals

In the four studies that reported patients' age and sex, the overall mean age was 58.4 years, with no significant difference between groups (weighted difference in mean ages 0.26 years, 95% confidence interval 1.25 to 1.77, χ2 test of between trial heterogeneity 3.36, P>0.3); 44.2% of the subjects were female. 14 17 19 20 Patients were free of “significant diabetic complications or serious medical conditions,” and only a minority of subjects (124, 12.2%) were treated with insulin. 14 19 20

The report of Porter's study provided data only on mortality, although it was stated that “no statistically significant differences could be demonstrated between the two groups (hospital and general practice care) in any of the biochemical or clinical indicators selected for measurement.”18

Meta-analysis

Metabolic control

Overall, there was no significant difference in metabolic control of patients receiving general practice and hospital care (fig 1; the weighted difference in mean glycated haemoglobin was 0.005% (0.26% to 0.25%). The mean glycated haemoglobin in the general practice group was equal to or less than that of the hospital group in all three studies that evaluated prompted care. 14 19 20 This heterogeneity between trials was confirmed by the χ2 value of 17.0 (P<0.001).

Mortality

Two patients in the Hoskins trial who died were excluded from analysis and their treatment group was not reported. A total of 84 patients died during the remaining studies, significantly more in the general practice group (odds ratio 1.75, 1.11 to 2.74) (fig 2). Mortality remained significantly higher in primary care even if the two patients from the Hoskins trial were assigned to the hospital care group. Mortality varied between studies (χ2=3.74, P>0.25), with most of the excess deaths in general practice care accounted for by the two earlier trials of care without prompting. 17 18

Losses to follow up

Patients randomised to general practice care were more likely to be lost to follow up (odds ratio 3.05, 2.15 to 4.33). This finding was accounted for almost entirely by the early study by Hayes,17 in which no organised system for recall was set up in general practice (χ2=114.8, P<0.0001), supporting a stratified approach to analysis (see below).

Hospital admissions

Only two studies reported usable data for hospital admissions: the earlier study favoured hospital care and the later one favoured prompted general practice care. 17 19 In addition, the diabetes integrated care evaluation (DICE) study, without reporting raw data, found no significant differences between prompted general practice and hospital care in unscheduled diabetes consultations or diabetes related hospital admissions.20

Blood pressure

Systolic and diastolic blood pressure did not differ between the prompted and hospital groups in two studies that included this variable (table 2). 14 20

Table 2

Summary weighted differences comparing prompted general practice care with hospital care

View this table:

Reviews and referrals

Two of the recent studies showed that patients were reviewed more often in the prompted group than in hospital outpatient departments and were tested more frequently for glycated haemoglobin (table 2) but were less likely to be referred to the dietitian (table 3). More chiropody referrals were made in the prompted group (table 3); however, as the two trials produced conflicting results for this outcome, the pooled data are unreliable. 19 20

TABLE 3

Summary odds ratios comparing prompted general practice care with hospital care

View this table:

Costs

Data on costs were not comparable between studies. Both studies that assessed costs borne by patients reported that they were lower in the community. 18 20 As regards the health service, basic general practice care cost about half as much as hospital care 14 18 ; prompted general practice care was found to be cheaper than hospital care by Hoskins14 but more expensive than hospital care in the DICE study.20

Stratified analysis

The studies fell into two categories based on the presence or absence of central, computerised prompted recall for patients and professionals in general practice. This division was supported by the heterogeneity of glycated haemoglobin results (fig 1) and mortality (fig 2). The between trial heterogeneity identified for each outcome when all trials were included in the analysis almost disappeared when trials of prompted care were considered separately (for glycated haemoglobin χ2=3.90, P>0.10; for mortality χ2=0, P=1.0; and for losses to follow up χ2=1.63, P>0.30).

Studies that incorporated prompting (Hurwitz,19 DICE,20 Hoskins 114) tended to produce outcomes favouring general practice care. Glycated haemoglobin values were lower (weighted difference in means −0.28%, −0.59% to 0.03%) and losses to follow up were significantly lower (odds ratio 0.37, 0.22 to 0.61) in prompted care but mortality was no different from that in hospital clinics (odds ratio 1.06, 0.53 to 2.11).

Sensitivity analysis according to the selection of denominator (either the number randomised or the number of subjects in whom the outcome was assessed) had no impact on the direction or significance of the effect sizes. The use of a random effects model affected significance only for comparisons with marked heterogeneity (mortality and losses to follow up when all trials were combined, and chiropody referrals).

Discussion

Meta-analysis based on a few small trials should be treated with caution, but this study suggests that prompted general practice care of people with diabetes, in certain circumstances, can be as good or better than hospital care. Furthermore, as losses to follow up were significantly higher from the hospital than from prompted general practice care and these defaulters tend to have poor outcomes, 21 22 the other comparisons may underestimate the achievements of prompted care in general practice.

Generalisability

The study population—patients who were happy to attend hospital clinics, had no diabetic complications or serious medical conditions, and were prepared to be randomised—represented between 38%19 and 88%20 of clinic attenders. Furthermore, the representativeness of the general practice participants was variable, from just three practices,20 through half,18 to almost all local practices becoming involved. 14 17 Thus conclusions from the meta-analysis should be generalised with caution. There is certainly little evidence to support the existing degree of general practice responsibility for diabetes care. Stratification of trials by the proportion of local practices involved did not explain interstudy heterogeneity.

Selection bias

The five trials were published in two of the journals that have been extensively hand searched. 23 24 The search strategy identified the same randomised trials and most non-randomised studies cited in a comprehensive systematic review including personal communication with authors.3 Thus publication bias and selective identification of positive studies are unlikely to explain the heterogeneity.

Sources of heterogeneity

Although stratifying by prompted or routine care explained the statistical heterogeneity, there are potential alternative explanations. The prompted care trials all reported in the past 5 years, so any variation could be a function of time. Fortunately, one recent study included both a prompted group (Hoskins 1) with better outcomes and a routine general practice care group (Hoskins 2) with rather poorer outcomes.14 Although the delivery of diabetes care by family doctors has undoubtedly improved over the past 20 years, time does not explain the heterogeneity and central prompting seems to confer additional benefit.

Heterogeneity could be due to the deterioration of hospital care since the early study by Hayes,17 but this is unlikely given that hospital care compared favourably with the routine general practice care arm of the Hoskins trial.14 There was little variation in the mean ages of the patients, and this did not influence the interpretation; neither did treatment at entry to the trial—studies including patients receiving insulin showed the effect of prompting. 14 20 Although the evidence that prompting explains the variation between studies is strong, the statistical tests for heterogeneity have low power, and residual unexplained variation in outcomes may still exist with a non-significant test.25

Other limitations of meta-analysis

The relationship between a complex, multivariate intervention (for example, introducing prompted care) and mortality or glycaemic control is unlikely to be linear; hence meta-analysis may produce imprecise estimates of effect size.26 In addition, data were extracted by just one author. However, it is reassuring that the findings are consistent with earlier reviews. 3 27 Perhaps the traditional literature review with meta-analysis for a few key outcomes provides a suitably balanced perspective. Furthermore, the heterogeneity between studies, and possible explanations for it, provide the main conclusion. Meta-analysis can magnify biases in original reports, and it attributes similar weight to findings from excellent and mediocre studies. This meta-analysis included all randomised studies to avoid subjective judgments of trial quality or arbitrary choices of an inclusion threshold based on explicit assessment criteria. 26 28

Limitations of the trials

One concern is whether these outcomes can be maintained in the long term. Clearly, 14 years ago in Cardiff, patients lacking regular prompted recall were worse after 5 years.17 As no other studies lasted longer than 2 years and the Hayes trial was responsible for much of the heterogeneity, this needs further research, as does the cost effectiveness of prompted care. Research should consider the potential duplication of care in the hospital and community and the balance between extra expenditure early on, as unmet need is identified, and longer term savings as expensive complications are avoided. 11 29 The extent to which the differences in process measures identified in this review influence the inexorable progress of diabetes remains unclear.30 Nevertheless, the variation in long term outcomes seen in these trials, and in other studies, 8 31 suggests that the organisation and delivery of care for this costly and increasingly common chronic disease are extremely important.32

The taxonomy and the appraisal of shared care are not fully developed; the studies tended to focus on clinical measures, hence the review is similarly constrained. Research is required to identify the important factors in successful shared care. Evaluating a complex process of managed change, such as the introduction of shared care, in terms of metabolic control only is limited, particularly for people with non-insulin dependent diabetes. Changes in health service delivery should achieve health gain, particularly reduction in risk factors for macrovascular disease; future studies should attempt to measure this.33 General practice care seemed to be popular with patients, 19 20 even if it seemed suboptimal by most objective measures.8 Some of these non-biochemical outcomes may prove to be more important than metabolic control in assessing how effective the systems of care are at helping people manage and adjust to their lifelong condition. 33 34

Acknowledgments

Acknowledgments

I am indebted to the Wellcome Trust for the support provided by my fellowship; Professor Rhys Williams, Dr Cathy Bennett, Ms Pat Spoor, and Dr Mark Airey (the Cochrane Diabetes Group) for helpful comments and assistance with database searching; and to Mr Andy Vail, Dr Paul Little, Professor Ann-Louise Kinmonth, Professor David Mant, the Cochrane Diabetes Group referees, and the BMJ referees for their helpful comments. I am grateful to Jane Overland, Mike Porter, and Simon Naji for helpful correspondence concerning three of the original trials.

Contributors: SG wrote the protocol, performed the searches (with help from Pat Spoor and Cathy Bennett), corresponded with original authors, extracted and analysed the data, wrote the paper, and is guarantor for the study.

Footnotes

  • Funding This study was funded as part of a Wellcome training fellowship in health services research.

  • Conflict of interest None.

References

Commentary: Meta-analysis is a blunt and potentially misleading instrument for analysing models of service delivery

  1. Trisha Greenhalgh, senior lecturer in primary care
  1. Primary Medical Care Group, University of Southampton, Aldermoor Health Centre, Southampton SO16 5ST
  2. University College London Medical School/Royal Free Hospital School of Medicine, Whittington Hospital, London N19 5NF

    The organisation of diabetes services in the United Kingdom is currently high on both clinical and political agendas. The disease is common (and getting commoner); the variability in current standards is dramatic and unaccountable; the evidence for health gain from systematic care is compelling; and the level of unmet need is probably unparalleled in any comparable condition. The service gap could, it seems, be competently filled by primary care, 1 2 and resources should naturally follow the patient.3 Some hospital diabetologists are palpably concerned about a service designed, delivered, and evaluated by generalists—but, in the era of the primary care led NHS, few are prepared to publish their reservations. The term “shared care,” with its nebulous connotations of the best of both worlds, is increasingly used in an attempt to square the circle.

    The cry for reallocation of resources is met, predictably and legitimately, with one for evidence of effectiveness. The stakes are high, but the evidence from published randomised controlled trials is scanty and of variable quality. The technique of meta-analysis, in which the results of separate clinical trials are summed mathematically, tempts us with the explicit promise of a level of objectivity, power, and precision that goes beyond that achieved in the individual component trials.4

    The result of Griffin's analysis, which is what some of us wanted to hear, is, broadly, that there is “no significant difference” in selected clinically relevant endpoints between structured care delivered in primary care and the same or a similar package delivered in the secondary sector. The conclusion, which I predict will go down in history as politics presented as science, is that “regular prompted recall and review of people with diabetes by willing family doctors” is achievable, is beneficial to patients, and should be supported by “suitable organisation.”

    Meta-analysis is inappropriate for trials which address different hypotheses, or which address the same hypothesis in very different ways.5 A high degree of statistical heterogeneity (measured by the χ2 statistic or an equivalent) is a necessary but not sufficient criterion for assessing the clinical importance of differences in inclusion criteria and methodology. Two apples and three oranges make two apples and three oranges, not five appleoranges, even if the individual fruits are the same size or weight.

    The intervention arms of the five trials shown in Griffin's figure 1 included two shifted care packages—patients were discharged empty handed in Cardiff6 and sent to private general practitioners with a hospital driven protocol in Sydney, Australia (“Hoskins 2”).7 Griffin contrasts these with three prompted care packages: patient-held checklists in Islington,8 divided (and, arguably, duplicated) care in Aberdeen,9 and nurse coordinated care (“Hoskins 1”) in Sydney.6 The last three models have in common the three Rs of successful structured care: registration, recall, and regular review, but their methodological differences (some but not all of which Griffin discusses) are more striking than their similarities.10 The aspects of care that contributed to the measured outcomes in each of these underpowered studies may thus be quite different, and the apparent increase in precision of the point estimate of effect may be illusory.

    We should be aware of the danger of false objectivity in a meta-analysis that draws together trials which are disparate, out of date, parochial, or plagued with practical limitations, particularly when no attempt is made in the analysis to weight them for generalisability or methodological quality.5 We should also note that randomised trials tend to attract not only a certain type of clinician but also a highly selected and atypical group of patients, who tend to be younger, less ill, and more accommodating than the general clinic population, and who virtually always speak the mother tongue of the investigator.11

    The conclusion from this diverse clutch of randomised trials in the methodologically challenging field of service delivery should therefore be cautiously drawn and modestly argued. Demands on the time and skills of primary care practitioners are high,12 and there is a deafening absence of evidence that the standards (such as they are) achieved in these subsidised short term studies could be achieved, let alone sustained, by “ordinary” primary care teams. Griffin acknowledges that things other than mortality and the biomedical dataset (notably, long term continuity of care and prevention of losses to follow up) are of paramount importance in the delivery of lifelong diabetes care, and others have argued persuasively that the quality of primary care must be measured by the tools of the humanities as well as those of evidence based medicine.13

    My own view is that in planning diabetes services we should use evidence, and meta-analysis where appropriate, not just to consolidate our knowledge but to face up to our ignorance and our uncertainty. Apart from the resounding failure of unplanned and unstructured care, the trials described above have raised more questions than they have answered about optimum organisational models for diabetes care.

    We still do not know, for example, the precise mix of competencies needed for delivering different aspects of education, surveillance, and support to people with diabetes. We do not know either the nature or the optimum time interval of the essential “routine” review. And we certainly do not know how best to communicate across interprofessional boundaries in so called seamless care. Although quantitative randomised trials can and should be conducted to address some of these issues, other issues will require primarily a social science rather than an epidemiological perspective.

    References

    View Abstract