http://orcid.org/0000-0001-8591-6103
Figures
Abstract
Introduction
The aim of this study is to develop a computer-aided diagnosis system to identify subjects at differing stages of development of multiple sclerosis (MS) using multifocal visual-evoked potentials (mfVEPs). Using an automatic classifier, diagnosis is performed first on the eyes and then on the subjects.
Patients
MfVEP signals were obtained from patients with Radiologically Isolated Syndrome (RIS) (n = 30 eyes), patients with Clinically Isolated Syndrome (CIS) (n = 62 eyes), patients with definite MS (n = 56 eyes) and 22 control subjects (n = 44 eyes). The CIS and MS groups were divided into two subgroups: those with eyes affected by optic neuritis (ON) and those without (non-ON).
Methods
For individual eye diagnosis, a feature vector was formed with information about the intensity, latency and singular values of the mfVEP signals. A flat multiclass classifier (FMC) and a hierarchical classifier (HC) were tested and both were implemented using the k-Nearest Neighbour (k-NN) algorithm. The output of the best eye classifier was used to classify the subjects. In the event of divergence, the eye with the best mfVEP recording was selected.
Citation: de Santiago L, Sánchez Morla EM, Ortiz M, López E, Amo Usanos C, Alonso-Rodríguez MC, et al. (2019) A computer-aided diagnosis of multiple sclerosis based on mfVEP recordings. PLoS ONE 14(4): e0214662. https://doi.org/10.1371/journal.pone.0214662
Editor: Joseph Najbauer, University of Pécs Medical School, HUNGARY
Received: November 21, 2018; Accepted: March 18, 2019; Published: April 4, 2019
Copyright: © 2019 de Santiago et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript.
Funding: This work was supported by Agencia Estatal de Investigación (AEI) European Regional Development Fund (ERDF) grant “Investigación de la técnica de potenciales evocados visuales multifocales. Aplicación en estudios de evolución de esclerosis múltiple y evaluación de medicamentos” reference: DPI2017-88438-R (AEI/FEDER,UE) to L.B., and RETICS OFTARED (Thematic Networks for Co-operative Research in Health for Ocular Diseases) RD16/0008/020 to L.B. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Multiple sclerosis and risk patients
Multiple sclerosis (MS) is a neurodegenerative disease characterized by chronic demyelination of the central nervous system (CNS) and which, as it develops, severely compromises patient quality of life. Although the cause of the disease remains unknown, it is assumed to be due to complex interactions between genetic and environmental factors. MS is not currently curable. The aim, therefore, is to diagnose it early and to provide treatment that reduces the risk of relapse and the progression of disability.
Radiologically isolated syndrome (RIS) defines individuals who do not show neurological symptoms but for whom magnetic resonance imaging (MRI) reveals findings that suggest future demyelination events. The conversion rate from RIS to MS was 65% after a mean follow-up of 5.3 years, and 88% after a mean follow-up of 14.1 years [1].
Clinically isolated syndrome (CIS) refers to a first episode of neurological symptoms that do not present fever and last a minimum of 24 hours. It is caused by inflammation or demyelination in the CNS. The most common symptoms are optic neuritis, a brainstem and/or cerebellar syndrome, a spinal cord syndrome or, occasionally, cerebral hemispheric dysfunction [2]. At median follow-up of 4.31 years, 623 out of 1047 CIS cases converted to MS [3]. Optic neuritis (ON) represents inflammation and demyelination of the optic nerve. In 38% of patients diagnosed with multiple sclerosis, ON is the first clinical manifestation of the disease [4].
Multiple sclerosis diagnosis
The current criteria used to diagnose forms of MS were originally formulated by [5] and revised by [6] and [7]. Diagnosis should take into account evidence of damage to the CNS disseminated in time (on different dates) and in space (damage to at least two different parts of the CNS) and should exclude other conditions that, due to their clinical or laboratory profile, can mimic MS.
Multifocal visual-evoked potentials (MfVEPs)
Multifocal visual-evoked potentials (mfVEPs) provide a method to diagnose optic pathway conditions. The visual stimulus is subdivided into a number (typically 60) of sectors. Each of these sectors is an independent stimulus controlled by specialized software. The electrical activity evoked by each stimulus in the visual cortex is recorded in the electroencephalograms (EEG). From a single, continuous EEG signal, a mathematical algorithm extracts the evoked response generated by each sector [8].
MfVEPs are a promising new tool of diagnostic and prognostic value in ON and MS [9]. MS leads to mfVEP abnormalities such as diminished intensity (amplitude), delayed nerve conduction velocity (latency) or morphological abnormalities and wave cancellation. MfVEP amplitude has been shown to be a functional biomarker of axonal loss in MS [10]. Latency is useful in assessing the extent of demyelination [11]. In [12], the authors demonstrated that mfVEP latency delay was evident in MS patients, where demyelination spreads along the entire visual pathway.
The various phases of record acquisition and processing have been enhanced to increase the effectiveness of diagnosis using mfVEPs. Previous papers have applied principal component analysis (PCA) to maximize sensitivity [13], Prony’s method to improve record quality [14], Gaussian wavelets to estimate latency reliably [15] and new types of stimuli to reduce test time [16]. Automated deep-learning image classification was used in [17] to analyse mfVEP plots; high AUC (area under the curve) and accuracy were achieved for assessing visual functions in patients with pituitary adenomas.
Computer-aided diagnosis (CAD)
In MS, early treatment has been considered the best strategy [18], hence qualitative and faster clinical decisions are needed. In line with this, computer-aided diagnosis (CAD) has become one of the most important areas of research [19].
Several papers have implemented CAD systems to analyse MRI images to improve MS diagnosis. A false-positive reduction method for analysing MRI lesions was proposed in [20]. A new method for analysing MRI images based on the wavelet transform and PCA analysis, with logistic regression as classifier, was presented in [21]. In [22], the authors diagnose MS versus controls, comparing three machine-learning-based classifiers: the decision tree, k-NN and the support vector machine using the wavelet and wavelet entropy obtained with MRI. More recently, [23] classified MS subtypes based on features gathered from MRI and the Expanded Disability Status Scale using non-linear classification models (Convex Combination of Infinite Kernels).
Other papers analyse the EEG using entropy parameters to distinguish between control subjects and MS patients [24] or perform classification using a multilayer perceptron artificial neural net of retinal-nerve fibre layer data [25].
Objective of this study
The objective of this study is to develop computer-aided diagnosis of multiple sclerosis based on mfVEP recordings. In the current study, a previously published patient database [10,26] is used, to which a new method of computer-assisted diagnosis is applied. The CAD system is implemented in two phases: in the first, the eyes are classified based on their mfVEP readings; in the second, diagnosis is performed on each subject.
Previous papers have demonstrated the existence of significant differences in the amplitude and latency parameters of mfVEP signals captured from control subjects and patients with differing levels of MS affectation [27]. MfVEP latency is potentially useful for assessing neuroprotective and remyelinating strategies in relapsing–remitting multiple sclerosis [28].
This paper uses the singular values of the mfVEP recordings, obtained using Singular Spectrum Analysis (SSA), as discriminating parameters in the eye classifier. The automatic classifier used is the k-Nearest Neighbour implemented on two separate structures: a flat multiclass classifier and a hierarchical classifier. Diagnosis is performed on each subject according to the results of classification of the subject’s eyes.
Materials and methods
Subjects database
The study protocol was approved by the Institutional Review Boards of University of Alcalá-affiliated hospitals and adhered to the tenets of the Declaration of Helsinki. The purpose and potential risks of the study were explained and all participants provided written informed consent. A cohort of eyes from patients with clinically definite MS (n = 56 eyes) and at different relative risk of developing MS, classified as RIS (n = 30 eyes) and CIS (n = 62 eyes), was included in this study and compared with a control group (n = 44 eyes) (Table 1). CIS and MS eyes were divided into two subgroups—ON eyes and non-ON eyes—based on whether or not they had had prior clinical ON episodes. ON episodes were always clinically defined by temporary reduced visual acuity in conjunction with other associated symptoms such as pain worsened by eye movement, variable visual field loss and colour perception changes.
All participants underwent an ophthalmologic examination that included best-corrected visual acuity (BCVA) using a high-contrast Snellen acuity chart. Inclusion criteria for RIS patients were based on [29]; MRI anomalies that did not account for clinically apparent impairments, and CNS white-matter anomalies with the following criteria: (1) ovoid, well-circumscribed and homogeneous foci with or without involvement of the corpus callosum; (2) T2-hypertensities measuring > 3 mm2 and fulfilling 3 of 4 Barkhof criteria [30] for dissemination in space; and (3) CNS anomalies not consistent with a vascular pattern. Eyes from CIS subjects having a first clinical episode suggestive of CNS demyelination involving the optic nerve, brainstem, spinal cord or other topography not attributable to other inflammatory diseases but lacking radiological evidence of dissemination of lesions over time were included in this study. Patients with CIS were included in the study within 3 months of their first clinical event. The MS groups comprised patients clinically diagnosed according to the McDonald criteria. MS patients had suffered one ON episode at least 6 months before being recruited. Full descriptions of the CIS and MS databases can be found in the following articles: CIS [26] and MS [31]. Finally, age-matched healthy subject eyes (n = 44 eyes) with normal neurological and ophthalmologic examination results were included as a control group.
MfVEPs
The practical aspects of taking mfVEP recordings have been described in previous papers [8,32]. Briefly, VERIS software 5.9 (Electro-Diagnostic Imaging, Inc., Redwood City, CA) was used to obtain 6 channels for each of the 60 sectors into which the visual field is divided. A sampling frequency of 1200 Hz was used and 600 samples were obtained in each recording (length 500 ms). The signals were digital-passband-filtered (1–35 Hz) using a fast Fourier transform. MfVEPs were always recorded outside acute relapse ON.
Obtaining the features of the mfVEP recordings
Amplitude.
The recording was divided into two different intervals: the signal window (45–150 ms), which contains the evoked potential response, and the noise window (325–430 ms), which essentially contains noise (Fig 1).
Definition of the signal and the noise window.
The amplitude is quantified as the signal-to-noise ratio (SNR) calculated as [33]:
(1)
where RMS(X45–150 ms) was the Root Mean Square (RMS) amplitude of the waveform in the signal window. The mean RMS(X325–430 ms) was the average RMS amplitude of all 60 waveforms in the noise windows.
The following calculations were performed using the channel with the highest SNR in each sector of the visual field (“the best channel”). If the SNR value of the best channel is under the threshold value of 1.7, the sector is considered a non-analysable sector (NAS) and it is discarded [33].
Latency.
Interocular latency [ms] is computed using the cross-correlation method. This method shifts the response from one eye along the x axis to maximum overlap (best correlation) with the response from the other eye. Hence, the amount of shift represents the latency difference between the eyes [34]. The computed interocular latency value was assigned for both eyes (OD and OS). Absolute values were used to obtain averaged interocular values. The sign (indicating whether OD or OS is longer) was ignored (S1 File).
Absolute monocular latency [ms] is obtained as the relative monocular latency of responses. The cross-correlation was calculated between the subject’s response and a template [35]. The template was created for each location, eye and channel and was derived from averaging the responses of 100 control subjects [36].
Singular spectrum analysis
Singular spectrum analysis facilitates time-series decomposition, trend detection, hidden component extraction, denoising and filtering. Several previous manuscripts present applications of SSA to analyse electrophysiological signals [37–40]. Singular values (σ1, σ2, σ3, …,σ7) obtained from SSA applied to mfVEP recordings were used to characterize the signal. Fig 2 shows the SNR values and the first 5 singular values (σ1, … σ5) in the 60 sectors of the visual field for the average of the entire control database. The singular values of a signal are approximately proportional to the mean value of a portion of the power spectrum. The lowest singular values (σ1, σ2 …) correspond to the lowest frequencies of the signal [41], so the first singular values contain trend information on the signal and the last singular values contain variance. The mathematical details of singular spectrum analysis are presented in S2 File.
When the values tend to zero, the colour of the sector is darker. R1,…R6: concentric rings.
Feature vector
The feature vector of each eye was computed according to a topographical division. The visual field (Fig 2) was divided into six concentric rings of increasing retinal eccentricity from 1° (R1: the most central ring) to 22.2° (R6: the most peripheral ring).
Based on the parameters described, the following feature vector is obtained for each eye (Table 2).
Machine-learning eye classification
The k-NN algorithm is a distance- and example-based non-parametric method, proposed by Cover and Hart [42]. In a classification problem, there is an M number of V vectors with features whose class membership is known: {Vj, Ch}, j = 1,…M; h = 1, …, H. When presented with a new feature vector (Vx), the objective of the classifier is to determine the class to which it belongs {Cx ∈ Ch}. The distance between Vx and the closest k vector Vj is calculated and the output class membership assigned is the most frequent in this set of k neighbours. If k = 1, Vx is assigned to the class of its nearest neighbour. The Matlab Classification Learner App (Mathworks Inc, Mass.) was used to train the model.
A flat multiclass classifier and a hierarchical classifier were tested (Fig 3).
F.V.: Feature Vector; ON: Optic Neuritis; non-ON: non-Optic Neuritis; RIS: Radiologically Isolated Syndrome; CIS: Clinically Isolated Syndrome; MS: Multiple Sclerosis.
The FMC input is the feature vector and the output comprises the 6 classes associated with differing degrees of development of MS into which the signals will be classified; implementation is achieved using a single k-NN algorithm.
The HC resolves the same overall classification problem in several classes using various appropriately ordered binary classifiers. It is implemented using 5 k-NN classifiers with varying k values and with different feature vectors.
The numbers of neighbours tested for each case are as follows: k = 1, k = 10 and k = all neighbours. The distance metric is Euclidean.
To assess classifier performance, cross-validation in n = 5 folds is used. Briefly, the cross-validation process is as follows: i) data are partitioned into n randomly chosen subsets (or folds) of roughly equal size, ii) one subset is used to validate the model trained using the remaining subsets. This process is repeated n times so that each subset is used exactly once for validation.
We use metrics that evaluate the classifier performance of each class (sensitivity, specificity and precision) and metrics that evaluate the overall performance: accuracy and Extended Matthew Correlation Coefficient (EMCC). EMCC is the generalization of the binary Matthew correlation coefficient to the multiclass case. It summarizes the confusion matrix. If the EMCC coefficient is equal to +1, classifier performance is perfect. If EMCC = -1, there is total disagreement between predictions and observations. The EMCC behaves consistently in practical cases and represents a good compromise between discriminancy, consistency and coherency with varying numbers of classes, unbalanced datasets and randomization [43].
Subject classification
Diagnosis is performed on each subject according to the following criteria: if both eyes of the same subject receive the same classification, that classification is assigned to that subject; if the eye classifier assigns a different classification to each eye of the same subject, the diagnosis will be that of the eye with the highest SNR.
Results
Feature vector
Table 3 shows the values of the parameters (SNR, NAS, interocular and monocular latencies and singular values) used as the feature vector for the various databases.
One-way analysis of variance (ANOVA) is used to compare the mean values of SNR, NAS, LATINTER, LATMONO, σ1,…, σ7 for the 6 different groups (CONTROL, RIS, CIS-non-ON, CIS-ON, MS-non-ON and MS-ON). The p-values obtained are less than 0.001 for all the parameters, except for σ4 (p = 0.0014) and σ5 (p = 0.0010); therefore, statistically significant differences between groups were found for each parameter, at the 0.05 level of significance.
Fig 4 shows the means plot with Least Significant Difference (LSD) intervals for each parameter by group, at the 95% confidence level. If two confidence intervals do not overlap, this offers evidence of a statistically significant difference between the means of the two groups, at the 0.05 level of significance.
SNR: Signal to Noise Ratio; NAS: Number of Analysable Sectors; LAT_INTER: Interocular latency; LAT_MONO: Monocular latency; ON: Optic Neuritis; non-ON: non-Optic Neuritis; RIS: Radiologically Isolated Syndrome; CIS: Clinically Isolated Syndrome; MS: Multiple Sclerosis.
SNR mean values decrease as risk of developing MS increases, with the lowest value being found in CIS-ON. Significant differences have been found in SNR values between controls and patients. No significant differences were observed between MS-ON and MS-non-ON eyes, because most non-ON eyes have been shown to be subclinically affected in clinically definitive MS groups [10]. Previous papers have found similar SNR values for MS patients with or without ON [44].
The lowest amplitude values were found in CIS groups. These signals were recorded within 3 months of the subject’s first clinical event, so the functional optic nerve fibres were still affected by axonal degeneration [45,46].
The lowest NAS mean value was found in the control subject group and increased as latency increased and amplitude decreased, with the highest value being found in the CIS-ON group. There is a significant difference between the NAS values of the control subjects and patients. A significant difference in NAS also exists between CIS-ON and the rest of the database.
In normal subjects, latencies in both interocular and monocular measurements should be essentially identical and close to 0 [47]. Low interocular values were obtained in the control group but were higher in patients. Significant differences were found between controls, RIS and CIS-ON. Interocular latency tended to be high in groups of patients with ON due to the functional differences between eyes in unilateral ON cases. Previous papers [48] found a significant increase in mfVEP latency in ON eyes compared to fellow (non-ON) eyes.
Monocular latencies were close to 0 in controls (-0.09 (4.94)) and non-ON eyes (CIS-non-ON: 0.59 (4.58) and MS-non-ON: 0.67 (5.59)). No significant differences were found between these 3 groups. High monocular latency values were found in ON-affected eyes (CIS-ON and MS-ON). A significant difference in monocular latency existed between patients with and without ON. This demonstrated that ON has a great impact on conduction velocity [10]. RIS patients presented moderate latency values and no significant difference with the other patient groups.
The singular values presented significant differences. There were no significant differences between the singular values of MS-non-ON and MS-ON eyes. σ1 σ2 σ3 performed similarly and discriminated between controls and RIS among ON patients (CIS-ON and MS-ON) and between CIS-non-ON and CIS-ON patients. σ4 σ5 σ6 σ7 likewise performed similarly and discriminated between the RIS group and the rest of the database.
Eye classifier results
Tests were performed using various parameters (k values, differing singular values for the signal) with both the FMC and the HC. The best results for each classifier were obtained with the configurations presented in Table 4.
The best value for nearest number of neighbours is 1, except for CLAS 5 (HC), in which it is 10. A low K value means that the best prediction is made when “local” information is used. K = 1 provides the most flexible fit, which will have low bias but high variance. A higher K value averages more voters in each prediction and hence it is more resilient to outliers.
The optimal singular values used in the feature vectors vary between the different classifiers from 0 (CLAS 1, CLAS 3) to 4 (CLAS 2).
To visualize the performance of the classifiers, the confusion matrix for the FMC and the HC are presented in Table 5.
HC delivers more correct values than the FMC (the values of the main diagonal in the HC are higher than those in the FMC). The FMC produces a high error rate when classifying RIS as CIS-non-ON (11 cases) and MS-ON as CIS-non-ON patients (18 cases).
The highest error rate produced by the HC occurs when classifying cases of CIS-non-ON as CIS-ON (10 cases) and MS-non-ON as MS-ON (13 cases); these classifier errors in ON diagnosis can be rectified using other specific diagnostic tests for this disease.
The numerical values used to evaluate the classifiers’ performance are shown in Table 6.
The parameters that reflect the performance of the HC exceed those of the FMC. This becomes clear when the average parameter values per class are analysed: mean(SensitivityHC) = 0.67 > mean(SensitivityFMC) = 0.45; mean(SpecificityHC) = 0.95 > mean(SpecificityFMC) = 0.90; and mean(PrecisionHC) = 0.70 > mean(PrecisionFMC) = 0.50. The same is the case for the overall parameters: AccuracyHC = 0.74 > AccuracyFMC = 0.54 and EMCCHC = 0.68 > EMCCFMC = 0.43. The only exception to this general performance affects specificity when detecting CIS-ON (SpecificityHC = 0.88 < SpecificityFMC = 0.97) and MS-ON (SpecificityHC = SpecificityFMC = 0.92).
Subject classifier results
Given that the best eye classification is obtained with the HC model, the output from that model is used to perform diagnosis on each subject. The confusion matrix for subject diagnosis is shown in Table 7.
For the controls in our database, classification is perfect: sensitivityC = specificityC = precisionC = 1. Of the 15 RIS-type subjects, one is classified as CIS, representing sensitivityRIS = 0.93, specificityRIS = 0.96 and precisionRIS = 0.82. In the CIS patient group (n = 28), 3 are classified as RIS-type subjects, representing sensitivityCIS = 0.90, specificityCIS = 0.97 and precisionCIS = 0.93. Finally, the results of MS patient classification are as follows: sensitivityMS = 0.96, specificityMS = 1.0 and precisionMS = 1.0. The overall parameters that reflect the characteristics of the subject classification are MCC = 0.93 and accuracy = 0.95.
Discussion
Although many biomarkers have been proposed for MS, at present only oligoclonal bands, magnetic resonance imaging, optical coherent tomography (OCT) and the JCv antibody index have been implemented in clinical practice [49]. MRI is the clinical test most widely used in diagnosing and monitoring MS [50], even though measures of white matter lesions do not correlate strongly with patients’ clinical symptoms (the MRI paradox [51]) and so ambiguous cases are frequently found in clinical practice [52]. For example, in [53] a modest correlation (r = -0.30) is obtained between MRI measurements of total brain white-matter lesions and cognitive function in MS patients.
Evoked potentials are considered to obtain better correlation values with the Expanded Disability Status Scale than structural data [54]. In addition, the most recent review of the McDonald criteria [7] recommends conducting studies of visual-evoked potentials in support of MS diagnosis. In line with this, the objective of this paper has been to advance implementation of a computer-aided system for diagnosing multiple sclerosis using mfVEP features and automatic classifiers that may help to address this clinical need.
Typical parameters used to study and quantify mfVEP recordings are signal amplitudes (related to axonal loss processes) and latencies (due to demyelination). These parameters show some capacity to discriminate between patient groups and identify significant differences in some cases when comparisons are made group by group. To the best of our knowledge, there have not been any prior studies into the relationship between the singular values of the mfVEP signals and the underlying electrophysiological processes. It has been shown that the singular values of the mfVEP signals provide discriminatory information that may be used to identify subjects with differing degrees of the disease.
This set of parameters (amplitude, latency and singular values) was used as input for a system that successfully discriminated between groups of eyes at different degrees of risk of MS.
Two different approaches were tested to implement the automatic eye classifier: FMC and HC. Parameters that reflect the performance of the HC exceed those of the FMC. This improvement may be because it adopts a divide-and-conquer approach to the problem by splitting the overall problem into smaller sub-problems. It also allows customization of various parameters of the intermediate classifiers: number of neighbours (K) and number of singular values used in the feature vector.
The relatively high EMCC value in HC (EMCC = 0.68) indicates that this classifier performs well. The specificity (0.95 mean value) is good for all the cases and higher than the sensitivity (0.67 mean value). This means that this classifier is better at detecting negative cases than at detecting positive ones.
Diagnosis of each subject is based on classification of both eyes. The overall parameters that characterize the confusion matrix (Table 7) are MCC = 0.93 and accuracy = 0.95.
Several previous papers have employed machine learning solutions to perform classification between controls and MS patients. A pattern recognition technique is used in [55] to learn a discriminant function that obtains a sensitivity of 0.82 and a specificity of 0.86 to distinguish between MS patients and controls using functional MRI (fMRI). In [56], SVMs are used as classifiers with diffusion tensor imaging (DTI) and fMRI data input between multiple sclerosis (RRMS) patients, obtaining accuracy = 89% ± 2%. In [25], measurements of the retinal nerve fibre layer (RNFL) are obtained. These are classified using a neural network to obtain AUC = 0.945 in the MS patients as compared with healthy subjects. Bayesian statistics-based biomarker creation was used in [57] to diagnose classes of either MS patients or controls according to alterations in bioactive lipid metabolism, achieving a sensitivity, specificity and accuracy of approximately 95% in training and test datasets. Recently, Ahmadi et al. [58] evaluated the phase–amplitude coupling of EEG signals in MS patients and controls; using an extreme learning machine neural network with online learning, they obtain accuracy = 0.912.
In this paper, classification between controls and patients is perfect: sensitivityC = specificityC = precisionC = 1. It is difficult to determine whether the advantage obtained over previous studies is due to the type of test (MRI, electrophysiology), to the type of machine learning applied or to the heterogeneity of the databases used.
Our study also classifies patients into 3 types: RIS, CIS and patients diagnosed with definitive MS. Of the RIS-type subjects, one is classified as CIS-type, representing sensitivityRIS = 0.93, specificityRIS = 0.96 and precisionRIS = 0.82.
For the CIS patient group (n = 28), 3 are classified as RIS-type subjects, representing sensitivityCIS = 0.90, specificityCIS = 0.97 and precisionCIS = 0.93. The 3 misclassified subjects are CIS subjects who have not suffered optic neuritis and whose potentials are largely unaltered.
Finally the results of classification of the MS patients are sensitivityMS = 0.96, specificityMS = 1.0 and precisionMS = 1.0.
To the best of our knowledge, very few papers have applied machine learning to classification of patients with differing levels of the disease. In [56], SVMs are used as classifiers based on diffusion tensor imaging (DTI) and fMRI data. Weak identification accuracies are obtained (63% ± 5%) when comparing MS patients with different levels of EDSS. Barbour et al. [59] develop diagnostic molecular-based (cerebrospinal fluid) classifiers. The classifier obtains the following AUC values for each possible classification: RRMS vs progressive MS: AUC = 0.91; MS vs CNS diseases that mimic MS: AUC = 0.98; relapsing–remitting vs MS: AUC = 0.91 and primary progressive vs secondary progressive MS: AUC = 0.5.
The main advantages of our method are that a) it obtains good values in patient diagnosis (Table 7), which evidently need to be confirmed by other clinical trials, and b) it is a fully automatic non-invasive method that does not require human intervention, as calculation of the elements of the characteristics vector and subsequent diagnosis is automatic. This avoids the need for tedious signal analysis by practitioners.
All these results show a promising machine-learning approach to identifying multiple sclerosis patients with a high degree of accuracy. Moreover, since the combined diagnostic technique (MRI, visual-evoked potentials, OCT) produces a huge amount of data it is beneficial to have access to machine-learning solutions that support practitioners’ decision-making.
Limitations and future work
RIS patients present higher singular values than controls in all the cases (σ1,σ2…, σ7). To the best of our knowledge, this was the first time that SVD was applied to mfVEP signals, so it was not possible to compare these results with other papers. In our opinion, these SVD values were not related to the presence of noise or artefacts because the SNR values were high (4.34 ± 1.77) and mfVEP signals are only considered artefacted or noisy if the SNR is below 1.7 [60]. Previous applications of SVD to other electrophysiological signals did not state this fact [41,61]. At this stage of the study, there is no clear hypothesis about the physical meaning of this result and future work must investigate these findings.
One possible improvement to the HC would be to add more classifiers, particularly to classifiers with low sensitivity and high specificity. For example, in classifier 5, eyes classified as MS-ON (Sensitivity = 0.28 and Specificity = 0.9) would be input into an additional classifier to increase accuracy. These additional classifiers could be based on other parameters (wavelet coefficients, entropy values).
This paper has only used the basic family of K-NN classifiers. Testing other types of complex classifier, such as neural networks or support vector machines, could improve the accuracy of the results.
The database used is small and taken from a single centre. To avoid bias deriving from the differences between databases, the database should be extended to include subjects from other centres and signals recorded using other commercially available equipment.
This new independent database will show the real diagnostic utility of the analysis method presented.
Due to the small size of the database, it has been considered advantageous to collect data from both eyes. This increases the number of cases in the study, but may decrease its statistical power [62] when classifying eyes. This is a limitation of this study that will be addressed in future work.
Perhaps the best enhancement, considering the scientific literature available, would be to use a different series of clinical data and MRI recordings justified by the results used in clinical settings to diagnose MS and by the publications referred to in the state of the art.
Conclusions
The objective of this paper has been to advance implementation of a computer-aided system for diagnosing multiple sclerosis using mfVEP features and automatic classifiers. In addition to amplitude (axonal loss) and latency (demyelination), it has shown that the singular values of the mfVEP signals provide discriminatory information. The best results for eye classification were obtained using a combination of K-NN classifiers and a hierarchical structure. In a second stage, a subject classifier based on the previous eye classification was implemented. The good results obtained with the subject classifier shows a promising machine-learning approach to diagnosing multiple sclerosis with a high degree of accuracy. This type of system can also be used to evaluate medical treatment response and can be employed in clinical trials.
Supporting information
S1 File. Appendix 1.
Interocular latency computing notes.
https://doi.org/10.1371/journal.pone.0214662.s001
(DOCX)
S2 File. Appendix 2.
Singular spectrum analysis.
https://doi.org/10.1371/journal.pone.0214662.s002
(DOCX)
Acknowledgments
Thanks to Dr Román Blanco (Universidad de Alcalá, Spain), for ceding the mfVEP recordings.
References
- 1. Okuda DT, Siva A, Kantarci O, Inglese M, Katz I, Tutuncu M, et al. Radiologically Isolated Syndrome: 5-Year Risk for an Initial Clinical Event. Jacobson S, editor. PLoS One. 2014;9: e90509. pmid:24598783
- 2. Miller D, Barkhof F, Montalban X, Thompson A, Filippi M. Clinically isolated syndromes suggestive of multiple sclerosis, part 2: Non-conventional MRI, recovery processes, and management. Lancet Neurol. 2005;4: 341–348. pmid:15907738
- 3. Kuhle J, Disanto G, Dobson R, Adiutori R, Bianchi L, Topping J, et al. Conversion from clinically isolated syndrome to multiple sclerosis: A large multicentre study. Mult Scler J. 2015;21: 1013–1024. pmid:25680984
- 4. Swanson JW. Multiple sclerosis: update in diagnosis and review of prognostic factors. Mayo Clin Proc. 1989;64: 577–86. pmid:2657239
- 5. McDonald WI, Compston A, Edan G, Goodkin D, Hartung HP, Lublin FD, et al. Recommended diagnostic criteria for multiple sclerosis: guidelines from the International Panel on the diagnosis of multiple sclerosis. Ann Neurol. 2001;50: 121–7. pmid:11456302
- 6. Polman CH, Reingold SC, Banwell B, Clanet M, Cohen JA, Filippi M, et al. Diagnostic criteria for multiple sclerosis: 2010 Revisions to the McDonald criteria. Ann Neurol. 2011;69: 292–302. pmid:21387374
- 7. Thompson AJ, Banwell BL, Barkhof F, Carroll WM, Coetzee T, Comi G, et al. Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. Lancet Neurol. 2018;17: 162–173. pmid:29275977
- 8. Hood DC, Greenstein VC. Multifocal VEP and ganglion cell damage: Applications and limitations for the study of glaucoma. Prog Retin Eye Res. 2003;22: 201–251. pmid:12604058
- 9. Pihl-Jensen G, Falck Schmidt M, Lautrup Frederiksen J, Schmidt M, Frederiksen JL. Multifocal visual evoked potentials in optic neuritis and multiple sclerosis: A review. Clin Neurophysiol. International Federation of Clinical Neurophysiology; 2017;128: 1234–1245. pmid:28531809
- 10. Blanco R, Pérez-Rico C, Puertas-Muñoz I, Ayuso-Peralta L, Boquete L, Arévalo-Serrano J. Functional assessment of the visual pathway with multifocal visual evoked potentials, and their relationship with disability in patients with multiple sclerosis. Mult Scler. 2014;20: 183–91. pmid:23828868
- 11. Alshowaeir D, Yannikas C, Garrick R, Van Der Walt A, Graham SL, Fraser C, et al. Multifocal VEP assessment of optic neuritis evolution. Clin Neurophysiol. International Federation of Clinical Neurophysiology; 2015;126: 1617–1623. pmid:25481337
- 12. Shen T, You Y, Arunachalam S, Fontes A, Liu S, Gupta V, et al. Differing Structural and Functional Patterns of Optic Nerve Damage in Multiple Sclerosis and Neuromyelitis Optica Spectrum Disorder. Ophthalmology. 2019;126: 445–453. pmid:30060979
- 13. Zhang X, Hood DC. Increasing the sensitivity of the multifocal visual evoked potential (mfVEP) technique: incorporating information from higher order kernels using a principal component analysis method. Doc Ophthalmol. 2004;108: 211–22. pmid:15573945
- 14. Fernández A, de Santiago L, Blanco R, Pérez-Rico C, Rodríguez-Ascariz JM, Barea R, et al. Filtering multifocal VEP signals using Prony’s method. Comput Biol Med. Elsevier; 2015;56: 13–19. pmid:25464344
- 15. Thie J, Sriram P, Klistorner A, Graham SL. Gaussian wavelet transform and classifier to reliably estimate latency of multifocal visual evoked potentials (mfVEP). Vision Res. Elsevier Ltd; 2012;52: 79–87. pmid:22100835
- 16. Horn FK, Selle F, Hohberger B, Kremers J. Steady-state multifocal visual evoked potential (ssmfVEP) using dartboard stimulation as a possible tool for objective visual field assessment. Graefe’s Arch Clin Exp Ophthalmol. 2015; 259–268. pmid:26553198
- 17. Qiao N. Using Deep Learning for the Classification of Images Generated by Multifocal Visual Evoked Potential. Front Neurol. 2018;9. pmid:30123181
- 18. Fernández O, Fernández V, Arbizu T, Izquierdo G, Bosca I, Arroyo R, et al. Characteristics of multiple sclerosis at onset and delay of diagnosis and treatment in Spain (The Novo Study). J Neurol. 2010;257: 1500–1507. pmid:20383518
- 19. Nayak DR, Dash R, Majhi B, Prasad V. Automated pathological brain detection system: A fast discrete curvelet transform and probabilistic neural network based approach. Expert Syst Appl. 2017;88: 152–164.
- 20. Yamamoto D, Arimura H, Kakeda S, Magome T, Yamashita Y, Toyofuku F, et al. Computer-aided detection of multiple sclerosis lesions in brain magnetic resonance images: False positive reduction scheme consisted of rule-based, level set method, and support vector machine. Comput Med Imaging Graph. 2010;34: 404–413. pmid:20189353
- 21. Wang S-H, Zhan T-M, Chen Y, Zhang Y, Yang M, Lu H-M, et al. Multiple Sclerosis Detection Based on Biorthogonal Wavelet Transform, RBF Kernel Principal Component Analysis, and Logistic Regression. IEEE Access. 2016;4: 7567–7576.
- 22. Zhang Y, Lu S, Zhou X, Yang M, Wu L, Liu B, et al. Comparison of machine learning methods for stationary wavelet entropy-based multiple sclerosis detection: decision tree, k -nearest neighbors, and support vector machine. Simulation. 2016;92: 861–871.
- 23. Karaca Y, Zhang Y-D, Cattani C, Ayan U. The Differential Diagnosis of Multiple Sclerosis Using Convex Combination of Infinite Kernels. CNS Neurol Disord—Drug Targets. 2017;16: 36–43. pmid:27781950
- 24. Torabi A, Daliri MR, Sabzposhan SH. Diagnosis of multiple sclerosis from EEG signals using nonlinear methods. Australas Phys Eng Sci Med. 2017;40: 785–797. pmid:28887746
- 25. Garcia-Martin E, Pablo LE, Herrero R, Ara JR, Martin J, Larrosa JM, et al. Neural networks to identify multiple sclerosis with optical coherence tomography. Acta Ophthalmol. 2013;91: e628–e634. pmid:23647619
- 26. Pérez-Rico C, Ayuso-Peralta L, Rubio-Pérez L, Roldán-Díaz I, Arévalo-Serrano J, Jiménez-Jurado D, et al. Evaluation of visual structural and functional factors that predict the development of multiple sclerosis in clinically isolated syndrome patients. Invest Ophthalmol Vis Sci. 2014;55: 6127–31. pmid:25190654
- 27. De Santiago L, Ortiz del Castillo M, Blanco R, Barea R, Rodríguez-Ascariz JM, Miguel-Jiménez JM, et al. A signal-to-noise-ratio-based analysis of multifocal visual-evoked potentials in multiple sclerosis risk assessment. Clin Neurophysiol. 2016;127: 1574–1580. pmid:26463474
- 28. Narayanan D, Cheng H, Tang RA, Frishman LJ. Longitudinal Evaluation of Visual Function in Multiple Sclerosis. Optom Vis Sci. 2015;92: 976–985. pmid:26267060
- 29. Moore F, Okuda DT. Incidental MRI anomalies suggestive of multiple sclerosis: the radiologically isolated syndrome. Neurology. 2009;73: 1714. pmid:19918002
- 30. Sastre-Garriga J, Tintoré M, Rovira A, Nos C, Río J, Thompson AJ, et al. Specificity of Barkhof Criteria in Predicting Conversion to Multiple Sclerosis When Applied to Clinically Isolated Brainstem Syndromes. Arch Neurol. 2004;61: 222. pmid:14967770
- 31. Blanco R, Perez-Rico C, Puertas-Munoz I, Ayuso-Peralta L, Boquete L, Arevalo-Serrano J. Functional assessment of the visual pathway with multifocal visual evoked potentials, and their relationship with disability in patients with multiple sclerosis. Mult Scler J. 2014;20: 183–191. pmid:23828868
- 32. De Santiago L, Sánchez-Morla E, Blanco R, Miguel JM, Amo C, del Castillo MO, et al. Empirical mode decomposition processing to improve multifocal-visual-evoked-potential signal analysis in multiple sclerosis. PLoS One. 2018;13. pmid:29677200
- 33. Zhang X, Hood DC, Chen CS, Hong JE. A signal-to-noise analysis of multifocal VEP responses: An objective definition for poor records. Doc Ophthalmol. 2002;104: 287–302. pmid:12076017
- 34. Hood DC, Zhang X, Rodarte C, Yang EB, Ohri N, Fortune B, et al. Determining abnormal interocular latencies of multifocal visual evoked potentials. Doc Ophthalmol. 2004;109: 177–87. pmid:15881264
- 35. Hood DC, Ohri N, Yang EB, Rodarte C, Zhang X, Fortune B, et al. Determining abnormal latencies of multifocal visual evoked potentials: a monocular analysis. Doc Ophthalmol. 2004;109: 189–199. pmid:15881265
- 36. Fortune B, Zhang X, Hood DC, Demirel S, Johnson C a. Normative ranges and specificity of the multifocal VEP. Doc Ophthalmol. 2004;109: 87–100. pmid:15675203
- 37. Kouchaki S, Sanei S, Arbon EL, Dijk D-J. Tensor Based Singular Spectrum Analysis for Automatic Scoring of Sleep EEG. IEEE Trans Neural Syst Rehabil Eng. 2015;23: 1–9. pmid:24951703
- 38. Hu H, Guo S, Liu R, Wang P. An adaptive singular spectrum analysis method for extracting brain rhythms of electroencephalography. PeerJ. 2017;5: e3474. pmid:28674650
- 39. Rasoulzadeh V, Erkus EC, Yogurt TA, Ulusoy I, Zergero SA. A comparative stationarity analysis of EEG signals. Ann Oper Res. 2017;258: 133–157.
- 40. Güven A, Altınkaynak M, Dolu N, Ünlühızarcı K. Advanced Analysis of Auditory Evoked Potentials in Hyperthyroid Patients: The Effect of Filtering. J Med Syst. 2015;39: 13. pmid:25637540
- 41. Bozzo E, Carniel R, Fasino D. Relationship between Singular Spectrum Analysis and Fourier analysis: Theory and application to the monitoring of volcanic activity. Comput Math with Appl. Elsevier Ltd; 2010;60: 812–820.
- 42. Cover T, Hart P. Nearest neighbor pattern classification. IEEE Trans Inf Theory. 1967;13: 21–27.
- 43. Jurman G, Riccadonna S, Furlanello C. A Comparison of MCC and CEN Error Measures in Multi-Class Prediction. Biondi-Zoccai G, editor. PLoS One. 2012;7: e41882. pmid:22905111
- 44. Sen P, Jayaraman M, Gandhi R, Ravi P. Multifocal visual evoked potential in optic neuritis, ischemic optic neuropathy and compressive optic neuropathy. Indian J Ophthalmol. 2014;62: 299. pmid:24088641
- 45. Klistorner A, Arvind H, Nguyen T, Garrick R, Paine M, Graham S, et al. Axonal loss and myelin in early ON loss in postacute optic neuritis. Ann Neurol. 2008;64: 325–331. pmid:18825673
- 46. Syc SB, Saidha S, Newsome SD, Ratchford JN, Levy M, Ford E, et al. Optical coherence tomography segmentation reveals ganglion cell layer pathology after optic neuritis. Brain. 2012;135: 521–533. pmid:22006982
- 47. Klistorner A, Fraser C, Garrick R, Graham SL, Arvind H. Correlation between full-field and multifocal VEPs in optic neuritis. Doc Ophthalmol. 2008;116: 19–27. pmid:17680288
- 48. Triplett JD, Yiannikas C, Barnett MH, Parratt J, Barton J, Graham SL, et al. Pathophysiological basis of low contrast visual acuity loss in multiple sclerosis. Ann Clin Transl Neurol. 2018;5: 1505–1512. pmid:30564617
- 49. Pulido-Valdeolivas I, Zubizarreta I, Martinez-Lapiscina EH, Villoslada P. Precision medicine for multiple sclerosis: an update of the available biomarkers and their use in therapeutic decision making. Expert Rev Precis Med Drug Dev. 2017;2: 345–361.
- 50. Kaunzner UW, Gauthier SA. MRI in the assessment and monitoring of multiple sclerosis: an update on best practice. Ther Adv Neurol Disord. 2017;10: 247–261. pmid:28607577
- 51. Barkhof F. MRI in multiple sclerosis: correlation with expanded disability status scale (EDSS). Mult Scler J. 1999;5: 283–286. pmid:10467389
- 52. Maggi P, Absinta M, Grammatico M, Vuolo L, Emmi G, Carlucci G, et al. Central vein sign differentiates Multiple Sclerosis from central nervous system inflammatory vasculopathies. Ann Neurol. 2018;83: 283–294. pmid:29328521
- 53. Mollison D, Sellar R, Bastin M, Mollison D, Chandran S, Wardlaw J, et al. The clinico-radiological paradox of cognitive function and MRI burden of white matter lesions in people with multiple sclerosis: A systematic review and meta-analysis. Aktas O, editor. PLoS One. 2017;12: e0177727. pmid:28505177
- 54. Giffroy X, Maes N, Albert A, Maquet P, Crielaard J-M, Dive D. Multimodal evoked potentials for functional quantification and prognosis in multiple sclerosis. BMC Neurol. 2016;16: 83. pmid:27245221
- 55. Richiardi J, Gschwind M, Simioni S, Annoni J-M, Greco B, Hagmann P, et al. Classifying minimally disabled multiple sclerosis patients from resting state functional connectivity. Neuroimage. 2012;62: 2021–2033. pmid:22677149
- 56. Zurita M, Montalba C, Labbé T, Cruz JP, Dalboni da Rocha J, Tejos C, et al. Characterization of relapsing-remitting multiple sclerosis patients using support vector machine classifications of functional and diffusion MRI data. NeuroImage Clin. 2018;20: 724–730. pmid:30238916
- 57. Lötsch J, Schiffmann S, Schmitz K, Brunkhorst R, Lerch F, Ferreiros N, et al. Machine-learning based lipid mediator serum concentration patterns allow identification of multiple sclerosis patients with high accuracy. Sci Rep. 2018;8: 14884. pmid:30291263
- 58. Ahmadi A, Davoudi S, Daliri MR. Computer Aided Diagnosis System for multiple sclerosis disease based on phase to amplitude coupling in covert visual attention. Comput Methods Programs Biomed. 2019;169: 9–18. pmid:30638593
- 59. Barbour C, Kosa P, Komori M, Tanigawa M, Masvekar R, Wu T, et al. Molecular-based diagnosis of multiple sclerosis and its progressive stage. Ann Neurol. 2017;82: 795–812. pmid:29059494
- 60. Hood DC. Multifocal VEP and ganglion cell damage: applications and limitations for the study of glaucoma. Prog Retin Eye Res. 2003;22: 201–251. pmid:12604058
- 61. Mahvash Mohammadi S, Kouchaki S, Ghavami M, Sanei S. Improving time–frequency domain sleep EEG classification via singular spectrum analysis. J Neurosci Methods. 2016;273: 96–106. pmid:27528379
- 62. Armstrong RA. Statistical guidelines for the analysis of data obtained from one or both eyes. Ophthalmic Physiol Opt. 2013;33: 7–14. pmid:23252852