Prospective evaluation of the capillaroscopic skin ulcer risk index in systemic sclerosis patients in clinical practice: a longitudinal, multicentre study

Systemic sclerosis (SSc) is a chronic connective tissue disease characterised by endothelial cell dysfunction and fibrosis of the skin and internal organs [1, 2]. Microangiopathy is one of the main histopathologic features detectable early in the course of the disease [3]. A gradual progression of vascular abnormalities has been observed during SSc progression [4]. Nailfold capillaroscopy (NC) is an imaging technique that detects morphological abnormalities of nailfold microcirculation. Furthermore, NC is an important tool for the classification and diagnosis of SSc in clinical practice [5, 6]. The three NC patterns early, active and late were found to be associated with Raynaud’s phenomenon (RP) as well as with the duration of the disease, possibly reflecting SSc evolution [4]. Although the diagnostic value of the NC patterns is well defined [7], different methodologies have been proposed to assess quantitative NC abnormalities in the follow-up of patients with SSc. However, their clinical applicability remains uncertain.

Sebastiani et al. [8] proposed the capillaroscopic skin ulcer risk index (CSURI) in 2009, as a quantitative measure of nailfold capillary damage that predicts the appearance of new digital ulcers (DUs) as well as the persistence of pre-existing DUs [8, 9]. The CSURI is based on the number of capillaries in the distal nailfold capillary row and the number of megacapillaries, as well as the maximum diameter of the megacapillaries on capillaroscopic evaluation [8, 9].

In order to gain better insight into the value of monitoring quantitative NC abnormalities in clinical practice, this multicentre study was designed to describe the reliability of the CSURI across different trained assessors, to describe the change of CSURI during follow-up, to assess the value of the CSURI in predicting new DUs and to assess associations between the CSURI and demographic and disease characteristics.

Between 2011 and 2015, 61 patients from eight centres were enrolled. The median observation time was 1.0 year (IQR 1.0–1.1). Of these 61 patients, 24 patients according to central assessor 1 (39%) and 26 patients according to central assessor 2 (43%) had no megacapillaries present on any assessed finger either at baseline or at the follow-up visit (Table 2). Due to the absence of megacapillaries, the CSURI could not be calculated for those patients. Therefore, for only 34 of the 61 eligible patients (56%) was the CSURI scorable by both central assessors at both time points. This percentage of patients without megacapillaries was similar across all eight centres (p = 0.72).

According to both central assessors, megacapillaries were present in 43 patients at baseline (Table 2); 30% of these showed an early SSc pattern on NC, 44% an active pattern and 26% a late pattern. Of the 18 patients without megacapillaries present at baseline (Table 2), 6% (one patient) had an early pattern, 28% a late pattern and the remaining 66% of patients showed no SSc specific pattern on NC at baseline.

The following analyses are entirely based on the 34 patients with an available CSURI by both central assessors at both time points, named the study population.

The baseline characteristics of the study population are presented in Table 3. The median observation time in this population was also 1.0 year (IQR 1.0–1.1). There were no statistically significant differences between the patients included in the study population and those excluded from further analysis. The included patients were, however, slightly younger (median age 57 years vs 62 years) and nominally more often had diffuse skin involvement (41% vs 33%) than the excluded patients. As many as 24% of the patients had experienced ulcer complications (soft tissue infections and gangrene).

In the study population, central assessor 1 counted a median of five capillaries in the distal row (range 2–10) and a median of one megacapillary (range 1–6) with a median maximum diameter of 62.5 μm (range 50–130 μm). Central assessor 2 counted a median of five capillaries in the distal row (range 2–10) and two megacapillaries (range 1–20) with a median diameter of 75 μm (range 30–180 μm).

The median baseline CSURI scores were 5.3 (IQR 2.6–16.3) as evaluated by central assessor 1 and 6.4 (IQR 2.4–12.5) as evaluated by central assessor 2. The median baseline CSURI was 8.2 (IQR 4.5–23.6) according to the local assessors. According to central assessor 1, the median CSURI score increased to 5.9 (IQR 1.3–12.0) at follow-up, whereas the median CSURI as evaluated by central assessor 2 decreased to 5.0 (IQR 1.7–10.0) at follow-up. The correlation coefficient between the baseline CSURI of the two assessors was 0.42, indicating a fair agreement [11]. There was a poor to fair agreement between the CSURI scored by the central assessors and the local assessors (central assessor 1/local assessors 0.45; central assessor 2/local assessors 0.38).

As evaluated by central assessor 1, 35% of patients had a higher CSURI at follow-up compared to 44% when evaluated by central assessor 2. In only 40% of the 34 patients was the change in CSURI between baseline and follow-up in the same direction for both central assessors; that is, an increase as measured by both assessors, a decrease in the measurements of both assessors or no change (Fig. 1).

According to both central assessors, 10 patients (29%) were in the low-risk category (CSURI < 2.96 [9]) at baseline; however, only seven of those 10 patients were concomitantly rated by both assessors as being in the low-risk category. According to the local assessors, five patients (15%) were in the low-risk category; however, only two of those were concomitantly rated by both central assessors into the low-risk category. The inter-assessor agreement of the central assessors for the baseline CSURI risk category was 0.58, indicating a moderate level of agreement [12]. The inter-assessor agreements of the local assessor and central assessor 1 or 2 were both 0.25, indicating a fair agreement.

Central assessor 1 scored 88% of the patients into the same risk category at baseline and at follow-up (i.e. either low–low risk or high–high risk); the remaining 12% were in the low-risk category at follow-up, but in the high-risk category at baseline. According to assessor 2, 73% of patients were in the same risk category at baseline and follow-up, 21% were in the high-risk category at baseline and in the low-risk category at follow-up, and 6% were in the low-risk category at baseline and in the high-risk category at follow-up. The agreement between the two central assessors of this ‘change in risk categories’ was fair (κ = 0.37) [12]. There was no agreement between central assessor 1 and central assessor 2 and the local assessors regarding this ‘change of risk category’ (κ = – 0.09, κ = – 0.16, respectively).

The prevalence of DU at baseline was 76% (Table 3) compared to 59% at follow-up. The ability of CSURI ≥ 2.96 (i.e. the high-risk category) to predict a higher number of DUs at follow-up than at baseline visit was rather low (positive predictive value for both central assessors 38%, for local assessors 48%), as was the ability of CSURI < 2.96 (i.e. low-risk category) to predict fewer or the same number of DUs at follow-up compared to baseline (negative predictive value for both central assessors 50%, for local assessors 67%). Out of the 34 included patients, 28 patients were classified into the same risk category by both central assessors. The positive and the negative predictive values based on these 28 patients were similarly lower (positive predictive value 38%, negative predictive value 43%) than the predictive values based on all 34 patients. The predictive values in patients who were treated with bosentan at any time during the observation period were similar to those who were not treated with bosentan.

No demographic or disease characteristic was associated with the change in the CSURI between baseline and follow-up simultaneously for both CSURIs, the one scored by assessor 1 and the one scored by assessor 2, in univariate linear regression (Table 4).

This prospective, longitudinal study examined the use of the CSURI in everyday clinical practice and demonstrates that 40% of patients in this multicentre study could not be evaluated with the CSURI at baseline and follow-up visits, mainly due to a normal NC pattern and the lack of any megacapillary as a prerequisite for the calculation of the CSURI [9, 13]. Additionally, the agreement of the CSURI between the two trained and experienced assessors was mediocre at best, as was the agreement between the two central assessors and the local assessors.

Our high percentage of non-scorable patients contrasts with the first CSURI study and the CSURI validation study [8, 9]. In the first study all patients had megacapillaries present, and in the second study only 13 out of an unselected SSc population of 242 patients (5%) were excluded from the study due to the absence of megacapillaries [8, 9]. However, in various other studies that were not applying the CSURI, the percentage of patients without megacapillaries was comparable to our high percentage. For instance, in a study of 188 SSc patients at least one quarter of patients had no megacapillaries [14]. Similarly, in two other studies, 24% and 30% of patients had no megacapillaries in any of the assessed fingers [15, 16]. Our discrepancies with the first CSURI studies are difficult to explain with differences of equipment, given the fact that very similar devices were in use in the first CSURI studies as well as in our study. Our patient population had similar disease duration as the patients consecutively recruited into the first CSURI study [8], but a higher proportion of diffuse SSc patients (41% vs 9%), which may not explain the lower prevalence of megacapillaries in our study.

In our study, the CSURI had only fair to moderate inter-rater reliability. This contrasts with an ‘almost perfect’ inter-observer reproducibility reported by Sebastiani et al. [8] in the original CSURI study, with κ = 0.96 based on the CSURI, dichotomised at the 2.96 cut-off value. A slightly lower but still ‘almost perfect’ inter-rater agreement of 0.85 was found in the validation study [9]. It is unlikely that these discrepancies can be completely explained with a lack of experience or different training, as both central assessors were trained together by authors of the original CSURI publications and used the same digital images and imaging software.

The CSURI was created as a prognostic index to predict the onset of new DUs [8]. In a validation study, Sebastiani et al. [9] demonstrated high predictive values for the development of DUs within 3 months, especially a high negative predictive value of 97%, but also a high positive predictive value of the CSURI of 81% in patients with a history of DUs. However, in another study by Sebastiani et al. [17] a poorer performance of the CSURI with lower predictive values was also observed in a population of SSc patients treated with bosentan. Differences in DU prediction may therefore be explained by differences in vasoactive medications. When we stratified our patients by bosentan treatment, we did not observe major differences in predictive values. It must, however, also be kept in mind that the predictive values from our study should not be directly compared with the studies by Sebastiani et al. as we assessed the predictive values of a higher number of DUs at follow-up compared to baseline and not ‘incident DU’ as Sebastiani et al. Additionally, the time between the baseline and the follow-up visit was considerably longer in our study (median time 1 year) than in Sebastiani et al.’s studies (3 months), which could also partly explain the differences in the predictive power of the CSURI.

A recent systematic literature review critically appraising studies reporting the prognostic value of NC in SSc also assessed the predictive value of the CSURI [18]. In line with our study, Paxton and Pauling [18] conclude that it is difficult to draw robust conclusions regarding the prognostic role of the CSURI; the reason for this being high levels of potential biases relating to study confounding as well as the statistical analyses.

It needs to be mentioned that our study has a rather limited sample size, which restricts the power to assess CSURI predictors in terms of demographic and disease characteristics. However, the mediocre performance of the CSURI regarding the inter-rater differences, as well as the high number of patients who could not be included due to the absence of megacapillaries, will not be a result of chance alone, even if a larger sample size would naturally have been beneficial.

The CSURI was not applicable in a large percentage of patients due to the absence of megacapillaries and demonstrated only fair to moderate inter-rater reliability. Thus, in routine clinical practice, the CSURI should be used with caution for treatment decisions and prediction of incident DUs.