Reliability of the evidence to guide decision-making in foot ulcer prevention in diabetes: an overview of systematic reviews

A librarian (MS) developed strategies to identify systematic reviews in Medline OVID and Embase OVID (initially from inception to December 2019 then re-run to update the searches until 17th May 2021) without restrictions. The first searches were de duplicated using RefWorks. The electronic search strategies were informed by the strategies reported elsewhere [11] and include methodological search terms (see Additional file 1: supplementary files). The Cochrane Database of Systematic Reviews (CDSR), the Database of Abstracts of Reviews of Effectiveness (DARE), and the Health Technology Assessment (HTA) Journals Library and (for the update search only, Epistomonikos) were also searched. Systematic reviews in progress were identified via PROSPERO (https://​www.​crd.​york.​ac.​uk/​prospero/​) and checked for subsequent completion or publication. Reference lists in all eligible reviews were browsed for additional relevant reviews. Additional data and clarifications about their reviews were sought from review authors.

Systematic reviews of RCTs of interventions to prevent foot ulceration in people with type 1 or type 2 diabetes whether at high, medium, or low risk, with or without a history of foot ulceration but no existing foot ulcers at baseline were eligible for inclusion. The outcomes of interest were incident primary or recurrent foot ulcers and Lower Extremity Amputations (LEA) derived from RCTs comparing single-component or complex interventions (comprising several interacting components provided together) with standard care or alternative interventions. We excluded reviews of surgical procedures. Systematic reviews that included RCTs and other study designs were eligible for inclusion but only data from the relevant RCTs was used for the purpose of the overview.

One reviewer (DJN or FC) screened all titles and abstracts to identify potentially relevant reviews with a second reviewer (FC or HMc) screening a 10% random sample to minimise the risk of errors of judgement. Reviewers working in pairs (DJN, AEA, FC or HMc) independently assessed the selected full text articles for eligibility and resolved disagreements in discussion with a third reviewer. Reviewers (DJN, AEA, FC or HMc) independently extracted data from the included reviews using a bespoke data extraction tool and resolved disagreements through discussion.

Reviewers working in pairs (DJN, AEA, FC or HMc) independently assessed the risk of bias in the included reviews using the Risk of Bias in Systematic Reviews (ROBIS) tool and reached agreement by discussion [12]. Concerns with the process of reviews are assessed using 4 domains; (i) study eligibility criteria, (ii) the identification and selection of studies, (iii) data collection and study appraisal and (iv) synthesis and findings.

Thirty-two reviews met the criteria for inclusion in the overview [13‐44]. Two were updates of previously published reviews and the earlier versions were excluded to avoid the double-counting of data [31, 33]. Of the 30 reviews, 14 included only RCTs [13‐26] and 16 included RCTs together with various other study designs [27‐30, 32, 34‐44]. The reviews were published between 1998 and 2021 in professional or scientific journals, four in the Cochrane Library, one was published in the UK National Institute of Health Research (NIHR) Health Technology Assessment (HTA) journals library and one for the Agency for Healthcare Research and Quality (AHRQ), USA [15, 16, 19, 20, 24, 41]. Other key characteristics of the included reviews are summarised in Table 1.

Overall, the 30 reviews included a total of 26 RCTs relevant to the overview (See Additional file 2: Supplementary references S1–S26). The majority of the RCTs were included in more than one review, only three being included only once (see Additional file 2: Supplementary references S16, S18, S20).

The ROBIS assessment results are summarised in Table 2. Six were judged to have a low risk of bias in all four domains assessed using the ROBIS tool [15, 16, 19, 20, 23, 24]. Nineteen reviews (65%) were judged to be at high risk of bias [13, 17, 18, 21, 22, 27‐29, 32, 34‐40, 42‐44]. The most common reasons for concern about bias in the reviews related to the lack of clarity in eligibility criteria specification (60%) [13, 17, 18, 21, 22, 27‐29, 35‐44] methods used to identify and select eligible studies (70%) [13, 14, 17, 18, 21, 22, 25, 26, 28, 29, 32, 34‐38, 40‐44] data collection and study appraisal (53%) [13, 17, 18, 21, 22, 27‐30, 32, 34, 36‐38, 42, 43] and the synthesis and findings (57%) [13, 17, 18, 21, 22, 25, 28, 29, 32, 35, 37‐40, 42‐44]. Only nine of the 26 non-Cochrane reviews reported the registration or existence of a review protocol [14, 19, 23, 24, 28, 30, 32, 39, 41]. The reviews used a variety of tools to assess the validity and risk of bias in trials with the Cochrane risk of bias tool being the most frequently used [6, 13‐18, 20, 23‐26, 29, 32, 41, 42]. Other assessment tools were, the JBL, [28, 39, 40, 45] reporting recommendations for trials of interventions for the foot in diabetes [27, 30, 46] PEDro [44, 47], the source of the risk of bias tool not reported [19, 22] QUADAS and other assessments for diagnostic tests [34, 37, 48, 49] Downs and Black [35, 50], McMaster Critical Review Form [38, 51]. Quality assessment not reported in two reviews [21, 43] preventative services veterans task force [36, 52].

Fifteen of the 26 non-Cochrane reviews either did not provide any information about sources of funding or declared none [17, 18, 22, 26‐30, 32, 35, 37‐39, 42, 43].

Eight reviews [20, 28, 32, 34‐36, 38, 40] aimed to evaluate footwear and/or offloading interventions and a further eight reviews of assorted interventions included footwear and offloading [14, 18, 19, 21, 22, 25, 30, 41], collectively identified nine RCTs relevant to the overview (Additional file 2: Supplementary references S5–S10, S16, S18, S19). Only two reviews were judged to be at low risk of bias [19, 20] and ten others were considered to be at high risk [18, 21, 22, 28, 32, 34‐36, 38, 40].

A Cochrane review published in 2000 [20] identified one quasi-randomised trial, in which patients were allocated alternately, not randomly, showed a significant reduction in recurrent ulceration with therapeutic shoes compared with standard footwear (Additional file 2: Supplementary reference S5).

Two subsequent reviews of the effectiveness of therapeutic footwear for preventing re-ulceration [34, 36] restricted inclusion of studies to those published in English, included one additional RCT (Additional file 2: Supplementary reference S6) and other study designs. The authors concluded that the evidence to support footwear interventions to prevent re-ulceration is conflicting because non-randomised and observational studies reported positive results while the RCT showed no benefit.

The quasi-randomised trial (Additional file 2: Supplementary reference S5) was the only study with an outcome relevant to the overview that was included in a review of the effectiveness of insoles for the prevention of ulcer recurrence [35]. This review considered evidence from mixed study designs which did not support its overly positive conclusions.

A review that focussed on the effectiveness of off-loading interventions to prevent primary (first) diabetic foot ulcers was restricted to studies published in English and failed to identify any relevant RCTs with ulceration as an outcome [28].

A review [32] (updating a previous version [33]) to inform IWGDF guidance on footwear and off-loading interventions to prevent and heal diabetic foot ulcers included five additional RCTs (Additional file 2: Supplementary references S7–S11). This review considered the findings from the RCTs (including the quasi-randomised trial) alongside results from cohort studies. The authors conclude that the evidence supporting the use of specific footwear interventions to prevent recurrent plantar ulcers is quite strong and that sufficient good quality evidence supports the use of therapeutic footwear with demonstrated pressure relief to prevent plantar ulcer recurrence [32]. This finding appeared to be based on the results from a subgroup analysis within a single RCT (Additional file 2: Supplementary reference S7).

A review of the effects of footwear and insoles published in 2020 [38] identified five RCTs, only one of which had not been included in a review previously (Additional file 2: Supplementary reference S19). The RCT (n = 51) compared ridged with semi ridged rocker soles in people at high risk of foot ulceration and found a statistically significantly reduction in ulcers in those allocated to the ridged rocker sole. The review concluded there was limited evidence to inform the use of footwear and insoles to prevent foot ulceration. A more recent review [40] included one RCT evaluating the use of a mobile phone to alert patients of increased foot pressures which was out with the search dates of all other systematic reviews (Additional file 2: Supplementary reference S18). The proof of concept trial allocated 90 patients who were at high risk of foot ulceration to an insole system where either audio-visual alerts via a smartwatch and offloading instructions were sent to the patients’ phones when increased pressures were detected or, in the control group, no alerts were sent. The trial had a large loss to follow up (36%) and no statistically significant difference in the number of ulcerations was observed but time to event analyses found the intervention group were ulcer-free for longer. The review concludes there was difficulty in singling out the most effective weight-redistributing preventative features in shoes and insoles but concluded that this type of intervention appears to be effective.

Eight other reviews of assorted preventative interventions were identified and again either included or pre-dated RCTs of footwear and/or offloading already described and identified no others [14, 18, 19, 21, 22, 25, 30, 41]. Meta-analyses of RCT data were presented in two of the more recent reviews [14, 25]. These suggest that footwear and insoles can reduce foot ulceration but further research to examine the most effective features of footwear and insoles and their effect in people with different risk profiles is merited.

We classified three systematic reviews of the effectiveness of interventions as complex [16, 37, 39]. One review was judged to be at low risk of bias [16], two others being judged to be at high risk. There were eight reviews of assorted interventions [14, 18, 19, 21, 22, 25, 30, 41] which included integrated foot care or complex interventions, and collectively all reviews included six RCTs relevant to the overview (Additional file 2: Supplementary references S1, S13–S16, S24).

A Cochrane review published in 2015 which assessed complex interventions defined as combinations of preventive strategies identified three RCTs relevant to the overview [16]. One RCT of an education-focused intervention in low to medium-risk patients (Additional file 2: Supplementary reference S12) reported a reduction in the incidence of foot ulceration compared with usual care but may not be reliable because the cluster-randomisation design was reportedly not accounted for in the analysis. One of two RCTs that compared more intensive and comprehensive complex interventions with usual care in high-risk patients showed no difference in the incidence of foot ulceration but a significant reduction in LEA (Additional file 2: Supplementary reference S13) whereas the other trial reported the opposite (Additional file 2: Supplementary reference S14). This review judged all three RCTs at high risk of bias and the pooling of data in a meta-analysis inappropriate due to marked heterogeneity. Overall, it concluded there was insufficient evidence to support the effectiveness of complex interventions.

A review of monofilament and other threshold tests for preventing foot ulceration was judged at high risk of bias across all 4 ROBIS domains and included only one RCT evaluating the prevention of foot ulceration and amputation in people with diabetes which was also included in the Cochrane review (Additional file 2: Supplementary reference S13) [37]. The review produced overly positive conclusions about the value of screening in preventing of foot ulcers and amputations given the trial found no statistically significant difference in the incidence of foot ulcers in the two groups [37].

The same trial was excluded from a review to inform IWGDF guidance on the prevention of foot ulcers in at-risk patients because of concerns about the comparability of the intervention and control groups [30].

The review undertaken to inform IWGDF guidance included studies of integrated foot care, defined as care given by one or multiple collaborating professionals treating patients on multiple occasions with multiple interventions [30]. It excluded the trial by McCabe et al. (Additional file 2: Supplementary reference S13) but included an RCT of chiropodist care (Additional file 2: Supplementary reference S15), (which was classified in other reviews as patient education) as well as unpublished data from an additional RCT of podiatric care (Additional file 2: Supplementary reference S16) which contributed to the assessment alongside data from non-controlled studies. No conclusion could be drawn about first ulcer prevention, and the suggestion that integrated foot care may be beneficial in preventing recurrent ulceration was largely based on the results from uncontrolled studies.

A systematic review of the effect of contact with a podiatrist, working within a team, on the incidence of foot ulceration did not identify any RCTs which met its own eligibility criteria [39].

The eight reviews of assorted interventions, details of which are presented below, identified no additional trials of complex interventions [14, 18, 19, 21, 22, 25, 31, 41].

The overview identified two systematic reviews evaluating telehealth interventions to prevent foot ulceration [42, 43]. Both reviews were judged to be at high risk of bias across all 4 ROBIS domains but only one included any RCTs. The review by Hazenberg et al. [42] analysed data from 4 RCTs of home-monitoring of foot skin temperature and presented a meta-analysis showing a reduction in the number of foot ulcers when abnormal temperatures were recorded and patients’ avoided weight-bearing until their foot temperature lowered (Additional file 2: Supplementary references S21–S23, S25). These same 4 RCTs were pooled by Ena et al. [26] in a review of temperature monitoring and were also included in three systematic reviews of assorted interventions [14, 25, 30].

The two reviews conclude that further research is required, [42, 43] one also acknowledge the limitations in the studies and that a larger evidence base is required before this technology could be widely implemented in practice [42]. However, the review by Ena et al. concludes that daily measurement of skin temperature when measured using a handheld infrared thermometer reduces the appearance of new foot ulcers and notes the risk of bias in the same 4 RCTs is low (Additional file 2: Supplementary references S21–S23, S25) [26]. The three reviews of assorted interventions all concluded that the available data suggest this intervention may prevent foot ulcers developing [14, 25, 30] but two noted the need for further evaluation and the possibility that the intervention might not be feasible in real world settings [14, 25].

We found one systematic review of physical activity which we judged to be at high risk of bias in its evaluation of the effect of exercise of the prevention of foot ulceration [44]. It included one RCT in which foot ulceration was an outcome (Additional file 2: Supplementary reference S24). The reviewers’ conclusion that exercise can delay the development of foot ulcers is not supported by the trial results (Additional file 2: Supplementary reference S24). The RCT was also included in three separate systematic reviews of assorted interventions [14, 30, 41].

Eight systematic reviews included a variety of interventions to prevent foot ulcers [14, 18, 19, 21, 22, 25, 30, 41] only one was judged to be at low risk of bias [19]. Four were judged to have an unclear risk of bias because of approaches they took to the selection of studies or the analysis [14, 25, 30, 41] and three were judged at high risk of bias [18, 21, 22].

Collectively they assessed the evidence from 26 RCTs, 2 of which were not included in intervention-specific reviews presented above. Two reviews included a trial of elastic compression stockings as a preventive intervention [14, 19]. The incidence of foot ulcers in people randomised to elastic compression stockings compared with those who did not receive hosiery was not found to be statistically significantly different. The trial population was at high risk of foot ulceration (Additional file 2: Supplementary reference S26).

Three reviews [14, 31, 41] included one RCT of patient instruction to apply antifungal nail lacquer as a way to increase the frequency of foot self-inspection but found no difference in the incidence of first or recurrent ulcers when compared with standard care (Additional file 2: Supplementary reference S17).