Article Text
PDF
Abstract
Introduction Visual evaluation of indirect immunofluorescence (IIF) on human epithelial-2 cells is the routine method for screening for antinuclear antibodies (ANA) in connective tissue diseases. Since visual IIF is time-consuming and subjective, automated IIF processors have been developed to offer standardised, valid and cost-efficient IIF assays.
Objective The aim of this study was to determine the diagnostic reliability of 2 widely used IIF processors (Aklides, Medipan GmbH and Helios, Aesku Diagnostics) under real-life laboratory working conditions.
Methods ANA were determined in samples from patients with suspected autoimmune rheumatic disease (n=1008) using both automated IIF processors and compared with the results obtained by visual interpretation. The performance of IIF processors to discriminate positive from negative samples, pattern recognition and end point titre prediction were evaluated.
Results The IIF processors showed moderate agreement with visual interpretation in discriminating positive from negative ANA samples (κ values: Aklides 0.494; Helios 0.415). The sensitivity/specificity was 89%/59% for Aklides and 87%/54% for Helios. However, both processors correctly identified 99% of definitely positive samples (titre ≥1:320). Aklides correctly identified 43% of fluorescence patterns and its light intensity values showed good correlation (Spearman's ρ=0.680) with visually obtained titres.
Conclusions Automated IIF determination under real-life laboratory working conditions remains a challenge. Owing to their high sensitivity at clinically relevant ANA titres, automated IIF processors can already support but not totally replace visual IIF.
- Autoimmunity
- Autoimmune Diseases
- Autoantibodies
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Key messages
What is already known about this subject?
Preliminary studies have shown promising results for automated indirect immunofluorescence (IIF) for antinuclear antibodies (ANA) determination in preselected sera.
What does this study add?
Under real-life conditions, automated IIF can reliably identify clinically relevant ANA titres and can therefore be a useful tool in laboratories.
How might this impact on clinical practice?
Since specificity and pattern recognition is unsatisfying, automated IIF currently cannot replace visual IIF.
Automated IIF determination under real-life laboratory working conditions remains a challenge.
Introduction
Autoimmune connective tissue diseases (CTD) are a heterogeneous group of systemic autoimmune diseases, including systemic lupus erythematosus (SLE), systemic sclerosis (SSc), Sjögren's syndrome (SjS), idiopathic inflammatory myopathies (IIM), mixed connective tissue disease (MCTD) and undifferentiated connective tissue disease (UCTD). These diseases are characterised by their specific autoantibody (AAB) profiles. Antinuclear antibodies (ANA) play a significant role and can be used for diagnosis, exclusion and monitoring of disease.1 ANA is a collective term for a large and heterogeneous group of circulating AAB. Reflecting their clinical importance, ANA are diagnostic or classification criteria for SLE, SSc, SjS, MCTD and UCTD.1–3
For the determination of ANA, a two-step approach has been established using visual indirect immunofluorescence (IIF) on human epithelial-2 (HEp-2) cells as an initial screening test and another specific immunoassay as a confirmatory test.2 ,4
The main advantages of ANA testing by IIF are the wide range of detectable antibodies, the high sensitivity and the possibility of simultaneously determining reactivity, titre and immunofluorescence pattern.5 Nevertheless, visual IIF has some substantial disadvantages. These assays require reading by experts, which is time-consuming and labour-intensive. Also, the correct pattern recognition depends on the individual qualification and experience of the investigator. Therefore, visual IIF is expensive and prone to interlaboratory and intralaboratory variability.6 ,7 There were intermediate intentions to substitute IIF with solid phase assays, such as ELISA, to reduce the burden of this process; however, these plans were abolished when unsatisfactory sensitivity was reported.8 Consequently, IIF has recently been confirmed by the American College of Rheumatology as the gold standard for ANA screening.9 As a result of this decision and an increased demand for ANA testing, automated IIF processors have been developed by the biomedical industry. Their intention is to offer standardised, cost-efficient and reliable IIF assays.10
Preliminary studies of these devices have shown promising results. Most of these studies, however, have focused on well-defined patient groups, rather than samples from routine practice. Consequently, it is unclear if these results are replicable in the clinical setting, where patient groups are less clearly defined and tests are ordered not according to recommendations.11 The aim of this study was to determine the diagnostic reliability of two IIF processors, Aklides (Medipan GmbH, Dahlewitz, Germany) and Helios (Aesku Diagnostics, Wendelsheim, Germany), under real-life laboratory working conditions.
Methods
Consecutive serum samples of n=1008 patients with a suspected autoimmune rheumatic disease (RD) were collected at the Department of Rheumatology and Clinical Immunology of the Charité Universitätsmedizin Berlin, and tested for ANA at the routine laboratory. Every serum sample was tested by visual IIF and two automated IIF processors, Aklides and Helios. The technicians were not blinded for the respective test results. The study was conducted with the approval of the local ethics committee (EA/193/10).
The majority of the patients were women (62%); their average age was 50.2 years (±19.5) with a range between 1 and 91 years.
Laboratories are not always informed about the final diagnosis. To analyse the relationship between correct diagnosis and obtained laboratory results, a selection of n=118 consecutive patients with a confirmed diagnosis from the Department of Rheumatology/Immunology at Charité was taken. These patients were divided into three subgroups by their respective diagnoses: non-RD with n=19, RD with n=99, including disease with ANA as diagnosis or classification criteria (DC) with n=12. The results of the two automated IIF processors were compared with the results obtained by visual IIF, which was defined as the current gold standard test.
ANA assessment by visual IIF
ANA were determined by visual IIF on the HEp-2 cell with commercial test kits (Generic Assays GmbH Dahlewitz/Berlin, Germany) according to the producer's instructions. For visual pattern recognition, a fluorescence microscope (Olympus AX70, Olympus Corporation, Tokyo, Japan) was used by trained experts. To exclude clinically irrelevant samples, titres with 1:160 were counted as positive, and those with 1:320 or higher were counted as strongly positive.12 ,13 According to the nomenclature of the International Consensus on ANA Patterns, only AC1-14 patterns with nuclear staining were considered positive. These patterns are also named by ICAP as ‘true ANA patterns’ in contrast to cytoplasmic or mitotic patterns.14
ANA assessment by automated IIF processors
Aklides is a semiautomatic IIF processor for reading of prepared ANA-IIF slides and capable of recognising the staining patterns. Positivity/negativity, the underlying fluorescence pattern and light intensity values correlating with the titre are determined using commercial test kits (Generic Assays GmbH Dahlewitz/Berlin, Germany). The Aklides system consists of a motorised scanning stage (Märzhäuser Wetzlar GmbH & Co. KG, Wetzlar, Germany), a fully automated fluorescence microscope (Olympus IX81; Olympus Corporation), a grey-scale camera, 400 and 490 nm light-emitting diodes (CoolLED, Andover, UK). A uniquely designed software system (Aklides) employs mathematical algorithms for pattern recognition and light intensity determination. Sera samples with a titre of ≥1:160 and a light intensity value of ≥100 were considered positive. Further details have been described in depth elsewhere.6
Helios is the first fully automated IIF processor that can perform all steps of IIF, including the preparation of slides and the positive/negative discrimination of ANA samples. Therefore, no further human intervention is necessary and users are offered true hands-off time. The Helios system discriminates between positive and negative ANA samples and presents relevant images to the user for each sample. Helios, however, cannot recognise staining patterns. In this way, the identification of specific fluorescence patterns is performed by the user. The system consists of barcode readers for complete traceability, a special three-needle system for fast pipetting operations enabling non-stop performance, a motorised and autofocus fluorescence microscope with NIKON optics and a specially designed software using mathematical algorithms for identification of ANA patterns. In this study, test kits by Aesku Diagnostics (Wendelsheim, Germany) were used. Sera samples with a titre of ≥1:160 and 2/3 or 3/3 positive evaluations were defined as positive. A more detailed description of Helios is available elsewhere.8
Statistical analysis
Cohen's κ was calculated for inter-rater reliability and Pearson's contingency coefficient as a measure of concordance. Contingency tables with visual IIF were used to determine sensitivity, specificity, correct classification rate, false classification rate, positive predictive values and negative predictive values. A receiver operating characteristic (ROC) with area under the curve (AUC) was conducted for light intensity values for Aklides. To determine correlation between end point titre and light intensity values of Aklides, Spearman's ρ was used. Statistical analysis was performed with SPSS Statistics 23 (IBM, Armonk, USA).
Results
Both automated IIF processors showed moderate agreement with visual IIF in discriminating positive from negative ANA samples (κ values: 0.494 for Aklides, 0.415 for Helios, Pearson's contingency coefficient: 0.457 for Aklides, 0.399 for Helios, p<0.001 for all). The performance of both systems, Aklides and Helios, compared with visual evaluation is shown in contingency table 1, respectively.
Contingency table of automated (Aklides and Helios) and visual antinuclear antibodies assessment
The sensitivity/specificity was 89%/59% for Aklides and 87%/54% for Helios, while the correct classification rate was 75% for Aklides and 71% for Helios. Aklides classified n=193 as false positive; in n=86 of these cases, cytoplasmic antibodies or centrosome patterns were identified visually.14 From the false negatives, 56 of 57 samples had a titre of 1:160 and 1 sample had a titre of 1:320. Of the n=220 false positives from the Helios system, n=90 samples had cytoplasmic antibodies or centrosome patterns. Almost all of the n=69 false negatives (68/69) had a titre of 1:160 and 1 sample had a titre of 1:320.
Of note, out of the strongly positive sera samples with a titre of 1:320 or higher, 165 of 166 were correctly classified as positive by both IIF processors. The discrepant serum sample was visually re-evaluated and classified as negative. In fact, 100% of strongly positive sera samples were correctly classified as positive by both IIF processors. Within the subgroup of DC, all 12 sera samples from patients with CTD were accurately identified as positive by both IIF processors.
The ROC curve (figure 1) confirmed Aklides to be a proper method for positive/negative discrimination of ANA with a good diagnostic accuracy (AUC: 0.851, p<0.001).
Receiver operating characteristic curve analysis of Aklides.
Concordance of automated and visual pattern recognition was analysed within n=118 randomly selected patients with known diagnosis. However, Aklides correctly assessed only 43% of the fluorescence patterns of ANA-positive samples. Pattern recognition ability was as follows: 1/1 centromere, 0/1 nuclear envelope, 15/39 fine speckled, 7/15 homogeneous, 4/7 nucleolar.
There was a good correlation between obtained light intensity values by Aklides and visually determined end point titres (Spearman's ρ: 0.680, p<0.001), allowing an estimation of the titre.
Discussion
Automated IIF processors have been developed to offer reliable, standardised and cost-efficient IIF assays. Preliminary studies in well characterised, rather small cohorts have shown promising results. In this study, the reliability of ANA assessment of two automated IIF processors was evaluated under real-life laboratory working conditions in a relatively large cohort of n=1008 consecutive sera samples.
It should be emphasised that 99% of the samples with a titre of 1:320 or higher, including all patients with confirmed connective tissue disease, were correctly classified by both IIF systems. Regarding all samples, however, both IIF processors showed only moderate inter-rater reliability with visual IIF. The pattern recognition of Aklides was correct in merely 43% of the cases. There was a good correlation between light intensity values and end point titres allowing an estimation of the titre.
Nevertheless, according to our results, the overall performance of the automated systems was acceptable but still can be optimised. While the observed sensitivity was satisfying (89%/87% for Aklides/Helios), the specificity was rather low (59%/54% for Aklides/Helios) and accountable for merely moderate inter-rater reliability. Since all false negatives of both IIF processors had a titre of 1:160, it can be assumed that the automated IIF systems have difficulties in differentiating between negative and weakly positive samples. In daily practice, however, low ANA titres are of minor clinical relevance, especially if ENA differentiation is negative.2 Cytoplasmic or mitotic patterns (centrosome) were detected in almost half of the false positives, perhaps indicating that IIF processors have difficulty distinguishing nuclear from cytoplasmic and mitotic patterns.14
Of note, our results differ from comparable publications on Aklides, Helios and other commercially available IIF processors.6 ,10 ,15–18 In a previous study, our group reported a κ value of 0.828, a contingency coefficient of 0.646 and a correct classification rate of 93% for Aklides.6 In a comparative study of six IIF processors, Bizzaro et al10 stated a sensitivity of 98%, a specificity of 85% for positive/negative discrimination, an AUC of 0.952 and Spearman's ρ of 0.672. Melegari et al19 documented a correct classification rate of 91%. Similarly, discrepant results to our study have been reported for the Helios system. Shovman et al described a higher inter-rater reliability with κ values of 0.633/0.657 for positive/negative results. Bizzaro et al reported a sensitivity of 98% and a specificity of 94%. Tozzoli et al published a correct classification rate of 98% for discrimination of positive from negative samples.
Presumably, the major reasons for these discrepancies are differences in study design, reference range and classification of results. In the study by Bizzaro et al, where preselected sera were used, only sera with high ANA titres and clear fluorescence patterns were investigated, which are much easier to distinguish from negative samples. In contrast to most comparable studies, this study used a reference range of 1:160 instead of 1:80. In our initial study, we separated results into negative, weakly positive and positive.6
Apart from Aklides and Helios, there are several other IIF processors commercially available, which also demonstrated good results in the study of Bizzaro et al.10 In agreement with the results of our study under real-life laboratory working conditions, automated IIF processors can reliably differentiate strongly positive from negative and weakly positive samples. However, at this stage, automated IIF cannot replace visual IIF, as ANA pattern recognition is not perfectly performed. Despite good correlation of light intensity values and titre, end point titres also cannot be predicted reliably. Recent developments, such as the improvements in pattern recognition and end point titre prediction, will prepare the gradual way for automated IIF into routine diagnostics.
Acknowledgments
The authors thank Mrs Mielke and Mrs Hinkel for their technical support.
References
Footnotes
EF and G-RB contributed equally.
Contributors CDL carried out immunoassays, performed statistical analysis, contributed with patients data and drafted the manuscript. KE conceived of the study, carried out immunoassays, participated in its coordination and helped to draft the manuscript. EF conceived of the study, participated in its coordination, contributed with patients data and helped to draft the manuscript. G-RB conceived of the study, participated in its coordination, contributed with patients data and helped to draft the manuscript.
Funding BMBF (German ministry of education and science); ArthroMark 01EG1401A.
Competing interests None declared.
Patient consent Obtained.
Ethics approval All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The study was approved by the local ethics committee (EA/193/10).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.