Field testing of the revised neuropathic pain grading system in a cohort of patients with neck and upper limb pain

2.1 Classification protocol

A detailed description of the patient cohort, previous clinical assessments and methodology is described in Tampin et al. [9]. Patients in the previous study had been recruited consecutively. The clinical assessment included the patient’s history, pain drawings, documentation of pain descriptors and pain behaviors, musculoskeletal assessments and neurological examination. Sensory testing comprised the assessment of light touch and pin-prick sensation in both upper limbs for determination of dermatomal sensory deficits and in both lower limbs, if spinal cord compromise was suspected, as well as in the maximal pain area for the assessment of NeP. Sensory testing findings were compared with the contralateral corresponding control site or, in case of bilateral pain, in proximal or distal pain-free sites [11]. Assessment of thermal sensitivity was not performed, consistent with previously documented methodologies [12], [13], [14], [15].

For the purpose of this study, the patients’ clinical notes were de-identified and electronically scanned by four postgraduate research students (JT, RB, LS, SG). Two highly experienced clinicians working in the field of NeP (RM, BT) independently reviewed the patient notes and imaging reports, a Pain Medicine Consultant (RM with 2 years of Consultant Practice) and an Advanced Scope Physiotherapist (BT; 9 years of Advanced Scope Practice and 26 years of musculoskeletal postgraduate experience) who had performed the initial patient assessment 8 years ago [9]. The latter had no recollection of the previous patient classification, however as an additional step to reduce bias, patient records were de-identified.

Prior to the independent clinical audit and re-classification, these two clinicians developed a recording sheet for the application of the revised NeP grading system [6] (Supplementary Table 1). Several randomly chosen imaging reports were reviewed in order to establish a consensus on which information would constitute a confirmatory test (Supplementary Table 2). To avoid false positives, diagnostic images were interpreted conservatively. Consistent with the previous study, only radiologists’ reports stating a compromise of the nerve root at the clinically-relevant level were deemed as a confirmatory test [9]. A pilot audit of 10 case notes was conducted prior to the full audit to allow consensus building between reviewers regarding the application of these criteria. A list of abbreviations used in the patient notes was compiled in order to allow clarity of interpretation of patient assessments (Supplementary Table 3). The 10 case notes and imaging reports in our pilot study were drawn from the patient sample used to establish consensus between clinicians. This method complied with recommended research strategies and consensus setting for retrospective audits [16].

Each clinician then independently reviewed the de-identified patient clinical notes, completed the recording sheets and classified the patients according to the revised NeP grading system. Audits were completed in batches of approximately 20–30 files at a time to ensure consistency. The first 20 completed cases were subsequently also re-evaluated by each reviewer at the end of the audit to ensure consistency. The clinicians were blinded from PD-Q and LANSS scores while classifying patients. Each clinician’s recording sheets were collected in a sealed envelope to maintain blinding prior to data analysis. The experts’ clinical diagnosis was then entered into an excel spreadsheet by the research students. Data entry was crosschecked among these students to ensure accuracy. This spreadsheet was merged with previously captured data on the two screening tools, PD-Q and LANSS questionnaires.

2.2 Clinical screening tools

2.3 Statistical analysis

The Statistical Package for Social Sciences (SPSS version 24.0, IBM, Armonk, NY, USA) was used to analyse the data. The demographic and clinical pain characteristics of patients were compared between four pain groups; no NeP, possible NeP, probable NeP and definite NeP, using Kruskal-Wallis and a one-way analysis of variance. Pairwise comparison of the following was performed:

1. Inter-rater agreement in classifying NeP according to the revised NeP grading system.

2. Clinical classification based on clinicians’ consensus and LANSS

3. Clinical classification based on clinicians’ consensus and PD-Q

The percentage of agreement, Cohen’s κ coefficient, the intraclass-correlation coefficient (ICC) and 95% CIs were calculated to determine the degree of concordance between the two clinicians in classifying the four categories of NeP (no, possible, probable, definite). The κ/ICC strength of agreement is classified as: >0=less than chance agreement, 0.01–0.20=poor agreement, between 0.21 and 0.40=fair agreement, between 0.41 and 0.60=moderate agreement, 0.61–0.80=substantial agreement and 0.81–1.00=almost perfect agreement [17].

A secondary analysis was performed to investigate the agreement in neuropathic pain classifications between the original and revised grading systems. Cohen’s κ with 95% CIs and the percentage of agreement were calculated.

To align with the dichotomous scale used in LANSS, dichotomous variables were applied for PD-Q (<19=unlikely NeP and ≥>19=likely NeP) and the clinical classification (no/possible=unlikely NeP; and probable/definite=NeP). The percentage of agreement, Cohen’s κ coefficient and 95% CI were calculated for the agreement between clinical classification and screening tools in classifying unlikely NeP and NeP.

The sensitivity and specificity of LANSS and PD-Q were calculated using the consensus of the expert clinicians’ clinical classification based on the revised grading system applied as the “gold standard”. Predictive values, odds and likelihood ratios were also calculated. The receiver operating characteristics (ROC) and the area under the curve (AUC) with their 95% confidence was computed for each screening questionnaire. A calculated probability of p<0.05 was considered significant for all analyses.

3.1 Patient characteristics

Out of the 117 patients meeting clinical consensus, none were classified as having no NeP (Tables 1 and 2). Thirty-nine patients (33.3%) were classified as having possible NeP, a large proportion of patients were classified as having probable NeP (n=66, 56.4%), while 12 patients (10.3%) were classified as having definite NeP (Tables 1 and 2). Table 2 illustrates patient demographics.

3.2 Inter-rater agreement on neuropathic pain classification

The two examiners agreed in classifying 117 out of 152 patients (ICC 0.794, 95% CI 0.716–850; κ 0.62, 95% CI 0.50–0.73) (Table 1), yielding a 77% overall agreement. The number of patients in each grouping based on the clinical classifications by the two independent clinicians is presented in Table 1. Out of the 117 patients, diagnostic testing results (such as imaging reports) were not available in 13 (11%) patients. Six of these patients were classified as having possible NeP, five patients as having probable NeP. The remaining two patients were classified by one clinician as having possible, and by the other clinician as having probable NeP, and vice versa. No patient with only localized neck pain, and an absence of sensory findings, was classified as having probable NeP.

In 35 cases (23.0%), no consensus was reached between clinicians (χ²=30.63, p<0.001) (Table 3). Diagnostic testing results were not available for two patients in the non-consensus group. There was a disparity in NeP classification group between both clinicians in 19 cases (16.2%) (Table 3). Out of the 19 cases, eight patients were classified under the “NeP” group by clinician 1 (RM), while these patients were classified under the “unlikely NeP” group by clinician 2 (BT). The other 11 cases were classified by clinician 1 (RM) under the “unlikely NeP” group and by clinician 2 (BT) under the “NeP” group.

Most of the non-consensus cases related to the classification between possible and probable NeP. In nine cases, clinician 1 (RM) classified the patients as having possible NeP, while clinician 2 (BT) classified them as having probable NeP. Six cases were classified by the clinician 2 (BT) as having possible NeP and were categorized as having probable NeP by clinician 1 (RM).

3.3 Agreement in neuropathic pain classification between the clinicians’ consensus and questionnaires where patients were classified as having NeP or no NeP

3.3.1 LANSS and the clinicians’ consensus

NeP classification differed between the LANSS and the clinicians’ consensus (χ²=8.72, p=0.002). LANSS identified 20 patients (17.1%) with NeP [mean 14.0 (range 12.0–17.0)] and 97 cases (82.9%) with unlikely NeP [mean 7.0 (range 0.0–11.0)] (Table 4). There was poor agreement between LANSS and the clinicians’ consensus in 57 out of the 117 cases (unlikely NeP, n=38; NeP, n=19; κ 0.159, CI 0.082–0.260), yielding 49% of agreement with 24.4% sensitivity and 97.4% specificity (Table 5). The predictive values, odds and likelihood ratios are listed in Table 5. Of the remaining 60 cases, 59 (98.3%) were classified as having unlikely NeP according to LANSS while clinicians’ consensus classified them as having NeP. The AUC curve for LANSS was 0.76 (95% CI 0.68–0.85; p<0.001) and the appropriate cutoff score for our population was 7.5 (Fig. 2).

Table 4:

Neuropathic pain (NeP) classification based on clinicians’ consensus, LANSS and painDETECT.

	Clinicians’ consensus		Total
	Unlikely NePa	Likely NeP
LANSSb
Unlikely NeP	38	59	97
Likely NeP	1	19	20
Total	39	78	117
painDETECTc
Unlikely NeP	26	37	63
NeP	13	41	54
Total	39	78	117

1. ^aNo and possible NeP grouped as Unlikely NeP. ^b49% agreement between clinical classification and LANSS. ^c57% agreement between clinical classification and painDETECT.

Table 5:

Accuracy of screening questionnaires in identifying patients with neuropathic pain.

	% Sensitivity	% Specificity	PPV	NPV	LR+	LR−	DOR
LANSS	24.4	97.4	0.95	0.39	9.38	0.78	12.0
PD-Q	52.6	66.7	0.76	0.41	1.58	0.71	2.2

1. PPV=positive predictive value; NPV=negative predictive value; LR=likelihood ratio; DOR=diagnostic odds ratio; LANSS=Leeds Assessment of Neuropathic Symptoms and Signs pain scale; PD-Q=painDETECT questionnaire.

Fig. 2:

Receiver operating characteristic (ROC) curve and area under the curve (AUC) of the Leeds Assessment of Neuropathic Symptoms and Signs pain scale (LANSS) and painDETECT questionnaire based on the revised grading system.

4.1 Main findings

To our knowledge, this is the first study to field test the clinical application of the revised NeP grading system in patients with neck and upper limb pain. A high inter-rater percentage agreement was demonstrated in classifying NeP. LANSS and PD-Q failed to detect a large proportion of patients with NeP.

The high inter-rater percentage agreement (77%) demonstrated in NeP classification indicates that the revised NeP grading system can be used reliably between skilled clinicians. The inter-rater agreement justified the use of the clinician’s consensus as the “gold standard” for a comparison with NeP screening tools and therefore enhanced the validity of the clinical NeP classification in this study compared to our previous study. While the percentage agreement between examiners was high, the κ value of 0.62 was just above the cut-off for substantial strength agreement, and the lower range of the confidence interval was within the moderate agreement range [17]. The κ coefficient can be a conservative estimate of agreement [18], [19] as this takes into account agreement occurring by chance. Given the application of a robust diagnostic work-up here, it is unlikely these classifications were based on guesswork. The ICC of 0.794 (95% CI 0.716–850) indicated substantial agreement between examiners and was close to almost perfect agreement.

Consensus was not reached between clinicians in 35 cases. In 16 of these cases (46%), patients were still classified by both clinicians into the same group: either “unlikely NeP” (no/possible) or the “NeP” group (probable/definite NeP) and therefore, would arguably have obtained the same clinical management [6]. Therefore, the disagreement was not necessarily clinically critical, as treatment recommendations would have been similar. For the remaining 19 cases (54%), discrepancies reflected different classifications into the “unlikely NeP” group or the “NeP” group. This is clinically important, as treatment approaches would differ according to these NeP classifications [6].

Reasons for disparity in classification were mostly due to differences between clinicians in the interpretation of sensory changes and imaging reports. Some patients had no evidence of sensory changes in their main pain area, however imaging reports were interpreted by one clinician as a confirmatory test and yet not by the other. The revised grading system proposes a hierarchical order of classification, hence in the absence of sensory changes the classification of probable NeP cannot be achieved as one cannot progress to the next step as this criterion has not been satisfied [6]. However, the authors of the revised grading system do comment in the legend of their Fig. 2, that in some cases sensory changes may be difficult to demonstrate, although the nature of the lesion is confirmed. For these cases, the classification “probable” would still apply if a diagnostic test confirmed a lesion [6]. While our clinicians did take these aspects into account, they still differed in their interpretation of some imaging reports. We propose that classification may be further improved where other clinical findings such as reflex and motor changes at relevant nerve root levels may be considered as a cluster of clinical findings that confirm a relevant lesion/disease.

Some imaging reports stated the presence of severe canal or foraminal stenosis, but did not comment on the presence or absence of neural compromise. In these cases, if the clinical presentation and clinical examination findings were consistent with a neural lesion, the imaging report could be interpreted as a confirmatory diagnostic test. In six cases, no consensus was met due to such ambiguity in interpreting patient clinical notes. These issues reflect the real world, as primary care clinicians rely on radiologist’s reports and a large proportion of imaging findings may be false positives [20] and possibly not directly relevant to treatment decisions.

Our observations suggest that in view of the heterogeneity in patients with neck and upper limb pain, a considerable amount of expertise is required to interpret the revised grading system. While the application was feasible in our clinical setting, it is unclear if this will be feasible to apply in primary health care settings where early recognition and timely intervention is often most needed.

4.2 Advantages and challenges of the revised grading system

The authors of the revised grading system have provided clearer guidance for clinicians on how to reach each level of certainty for the presence of NeP; i.e. taking into account pain descriptors for the classification of possible NeP and specifying the type of sensory change (negative sign) indicative of a nerve lesion. While the hierarchical order of classification may have clinical utility for non-specialists, this approach may lead to a false negative classification and potentially a lack of appropriate treatment. For example, a patient presenting with a C6 radicular pain distribution of symptoms and NeP pain descriptors, absent biceps reflex and C6 myotomal weakness, but without any sensory changes in the main pain area would be classified as having possible NeP. According to NeuPSIG guidelines pharmacological treatment for NeP would not be commenced [6] even though symptoms of NeP and signs of a nerve root lesion/radiculopathy are apparent. While experienced clinicians would not rely solely on such a classification system to make clinical decisions, this is not the case for primary care settings where decision support is often most needed.

4.3 Comparison of clinicians’ consensus and screening questionnaires

Compared to the “gold standard”, LANSS and PD-Q failed to detect a large proportion of patients with NeP, similar to results in our previous study [9]. Previously, LANSS demonstrated a sensitivity of 22% and specificity of 88% while PD-Q demonstrated a sensitivity of 64% and specificity of 62% [9]. The low sensitivity of LANSS (24%) could be attributed to the differing patient characteristics in our cohort compared to the cohort in the original validation study [9]. The specificity of LANSS was high, consistent with other studies [21], [22], indicating its usefulness in negating the presence of NeP in patients with neck-arm pain. The PD-Q demonstrated a sensitivity of 53% which was lower compared to our previous study and others [9], [23], and a specificity of 67% which had improved by 5% compared to previous findings [9]. The specificity may have improved as a larger proportion of patients were clinically classified as having unlikely NeP (n=39) compared to our previous study (n=28). Nevertheless, the discriminative ability of PD-Q in identifying NeP remains questionable.

It is unclear why LANSS performed better than PD-Q with respect to specificity. A comparison of the two screening tools is complicated by the fact that they differ in their design, their pain descriptor items and the scoring of these items. It is possible that the clinical examination tests of LANSS, which corresponded with the sensory testing used by the clinician, improved specificity, but not sensitivity.

Both screening questionnaires demonstrated insufficient diagnostic accuracy, consistent with findings in other studies [7], [24], [25]. This observation is further reflected in the lowered overall agreement between the screening tools and clinical classification here compared to our previous study (LANSS 49% vs. 68%; PD-Q 57% vs. 63%). Clinicians should be cautious about relying too heavily on these tools to detect the presence of NeP. Screening tools can supplement clinical reasoning, but cannot replace it.

4.4 Strengths and limitations

The strength of our study lies in the robust methods, using two independent clinicians’ rating, blinded application of the grading system and blinded data entry and analysis. A limitation is that not both clinicians examined the patients explaining some interpretation differences, although this occurred in only six cases. While a study design would be strengthened by both clinicians examining the patients, this adds considerable responder burden for patients and clinicians. We acknowledge the limitation associated with using the statistical assumption that the four NeP grades are mutually exclusive, and the distinct separation between possible and probable NeP into “unlikely NeP” and NeP. However, this is the design of the revised grading system and we applied this approach as a pragmatic decision. This highlights that in the real world, rather than the standalone application of a “grading system”, clinical decision making is optimally guided by all relevant clinical findings to ensure best practice care.

4.5 Conclusion

In conclusion, the application of the revised NeP grading system was feasible in our retrospective analysis of patients with neck/upper limb pain. High inter-rater percentage agreement was demonstrated. The feasibility and reliability of the grading system in primary health care settings needs further explorations and strengthening in decision support to ensure “right care”. The diagnostic accuracy of LANSS and PD-Q in identifying NeP in patients with neck/upper limb pain remains limited. Clinical judgment remains crucial to diagnosing NeP in the clinical practice.