EECoG-Comp: An Open Source Platform for Concurrent EEG/ECoG Comparisons—Applications to Connectivity Studies

This dataset is partly described in Nagasaka et al. (2011) and Oosugi et al. (2017). All the experimental and surgical procedures were performed in accordance with protocols approved by the RIKEN ethical committee (No. H22-2-202(4)) and the recommendations of the Weatherall report, “the use of non-human primates in research”. During the experiment, a macaque monkey (Macaca fuscata) was seated in a primate chair with hands tied and head movement restricted. The eyes of the monkey were covered to avoid evoked visual responses during the entire experimental period.

The electrophysiological data (EEG and ECoG) was recorded during two conditions: (1) awake resting state with eyes folded and movements restricted; and (2) drug induced anesthesia: 1.15 ml (8.8 mg/Kg) ketamine hydrochloride and 0.35 ml (0.05 mg/Kg) medetomidine injection. Two five-min trials of simultaneous resting state EEG and ECoG activities were recorded for each condition.

For ECoG, the skin, skull and dura of the monkey’s left hemisphere was carefully removed. Then, a modified human subdural ECoG array, adapted to the monkey physiology (Nagasaka et al. 2011), was inserted in the subdural space between the arachnoid membrane and the dura mater (see Fig. 2). This array was fixed on a set of 1 mm-thick silicone stripes, containing 128 insulated ECoG platinum disk-shaped electrodes with 3 mm in diameter and leaving only a hole in the silicone of 0.8 mm diameter at the center of each electrode. The electrodes, with a separation of 5 mm between each other, covered the frontal, parietal, temporal and occipital lobes and part of the medial wall of the left cortex, as depicted in the X-ray image in Fig. 3. The X-ray images were obtained from a VPX-30E system (Toshiba Medical Supply, Tokyo, Japan) with 8.0 mA in current and 90 kV in voltage. The reference electrode were rectangular platinum plates in the subdural space between the array and the dura. Another platinum electrode was placed as ground in the epidural space.

After placing the ECoG electrodes, the dura, the skull and the skin were carefully placed back (see Fig. 2). ECoG was filtered and recorded by a Cerebus recording system (Blackrock Microsystems, Utah, USA). Plastic connectors from the array to the ECoG system were attached to the top of the head with dental cement and small titanium screws. The sampling rate was 1000 Hz, and the recordings were Butterworth bandpass filtered within 0.3–400 Hz.

The EEG was recorded with a NeuroPRAX system (eldith GmbH, Ilmenau, Germany) of sampling rate 4096 Hz, using 18 electrodes which follows the standard 19-electrode 10–20 system. Electrode “Cz” has been removed to place the ECoG plastic connector. The recordings were down-sampled to 1000 Hz. An external TTL pulse was used to define the beginning and the end of the EEG and ECoG recordings interval of interest for synchronization.

The first challenge of analyzing the EECoG (MDR) data is the high level of artifacts present in the recordings. We therefore addressed this attention to assimilating procedures to alleviate this interference.

The first preprocessing step of the simultaneous EECoG recordings is synchronization. This is done by aligning the original 4096 Hz EEG and 1000 Hz ECoG recordings to the “Trigger” signal. After that, the “Trigger” channels are removed from both recordings. Then, we average referenced the recordings and remove the mean. Based on the observed EEG artifact properties of sparse spikes and step-like discontinuities, we further use Transient Artifact Reduction Algorithm (TARA, the theory will be detailed in Appendix A2) (Selesnick et al. 2014) to clean these artifacts, the results are shown in Fig. 4 below. We further apply the notch filter to remove the power-line artifact of 50 Hz and applied the Butterworth high-pass filter of 0.3 Hz. At last, we down-sampled the EECoG recordings to 400 Hz and bandpass to alpha band (8–12 Hz) for latter analysis. Our platform also supports FieldTrip (Oostenveld et al. 2011) and EEGLab (Delorme and Scott 2004) data structure, which allows the users to create their own pipeline out of this platform.

By introducing TARA, we were able to clean most of the above two types of artifacts as shown in Fig. 4. After preprocessing, several channels in the EEG (F3, C3, and Fz) and ECoG (channel 38, 50, and 93) have been removed from further analyzing due to bad quality, as visually judged by an expert neurophysiologist.

It is to be noted that, surprisingly, we did not observe a clear alpha peak, not even from the occipital recordings of both EEG and ECoG, despite our careful artifact rejection and pre-processing.

The second challenge of simultaneous EECoG data analysis comes from the implanted ECoG stripes. This makes the forward modeling procedure much more difficult. EEG forward modeling is the first step for ESI analysis, and it can provide the so-called lead field or gain matrix \(K\) for further source activity estimation and simulation. Thus, EECoG_FWM, as one important module of EECoG-Comp, is created to build the realistic head model for the complex simultaneous EECoG experiment. The EEG forward problem can be formulated as follows:

While the ECoG forward problem can be written as:where \( {\varvec{J}} \) is the primary current density, \( {\varvec{V}}_{eeg} \) and \( {\varvec{V}}_{ecog} \) are the measured EEG and ECoG signals, \( {\varvec{K}}_{eeg} \) and \( {\varvec{K}}_{ecog} \) are the lead fields for EEG and ECoG, \( \varvec{\varepsilon}_{eeg} \) and \( \varvec{\varepsilon}_{ecog} \) are the measurement noises for EEG and ECoG, respectively.

The aim of EECoG_FWM module is to estimate \( {\varvec{K}}_{eeg} \) and \( {\varvec{K}}_{ecog} \) from the T1-weigthed Magnetic Resonance Image (MRI) of the subject (in this data set, it is the only monkey named Su), these images were used to construct the surfaces (triangular meshes) defining the head model. The T1w image was acquired in a 4T Varian Unity Inova MR scanner (Varian NMR Instruments, Palo Alto, CA) with a three in one loop surface coil (Takashima Seisakusho Ltd., Tokyo, Japan). The scanning parameters were: TR/TE = 13 ms/3.8 ms, in-plane matrix of \( 256\; \times \;256\; \times \;256 \), field of view (FOV) of \( 12.8\; \times \;12.8\; \times \;12.8\,{\text{cm}} \) with voxel size of \( 0.5\; \times \;0.5\; \times \;0.5\,{\text{mm}} \). This image was segmented into white matter (WM), gray matter (GM) and cerebrospinal fluid (CSF) with BrainSuite2 (Shattuck et al. 2001). The gray and white matter interface surface, as shown in Fig. 5a, is chosen as the source space model for EECoG, i.e. each of its 104650 nodes can be the location of a current dipole with orientation normal to the surface, and we are using a down-sampled version in latter experiments due to the computational issues. The volume conductor model comprises three compartments (1) WM + GM + CSF, (2) Skull and (3) Skin as shown in Fig. 5a with constant conductivities 0.33 S/m, 0.0041 S/m and 0.33 S/m, respectively. These are represented by their boundaries: the “inskull”, the “outskull” and the “scalp”. Additionally, a fourth compartment accounting for the ECoG silicone stripes were included, with 1 mm thickness and the shape shown in Fig. 5b, and they are also described in the first subsection named “MDR Recordings”. It is very important to account for this compartment because its low conductivity and therefore it influences the EEG signals. On the other hand, the EEG electrodes were manually located on the monkey’s scalp using IMAGIC software (www.​neuronicsa.​com) as depicted in Fig. 5c.

The tetrahedral meshes in Fig. 5a were created from the surfaces of the head model using TetGen 1.5.0 (Si 2015), an open source software in Linux repositories. During the meshing process the nodes in the source space surface (cortical grid) were forced to be nodes of the tetrahedral mesh to guarantee the Venant’s condition. Both EEG and ECoG lead fields were calculated using NeuroFEM. This is a program for computing lead fields using FEM which is part of the SimBio software package (Wolters et al. 2007) (index page: https://​www.​mrt.​uni-jena.​de/​simbio/​index.​php/​Main\_​Page). This module can provide average referenced LFs based on 3d cortical grid source space.

It should be noted that only 15 channels of EEG left after removing the bad channels, and 125 channels of ECoG left. Finally, we only took the left hemisphere of the cortical grid (source space) and study its connectivity due to the limited coverage of ECoG in real data analysis.

Brain connectivity has received much attention in recent years (Bassett and Sporns 2017). However, the estimation and comparison of brain connectivity at source level is still very challenging (Zalesky et al. 2010). The next challenge for EECoG-Comp platform is to compare the source level connectivity with sound statistics. EECoG_conComp is designed to fulfill this goal, and it has the following submodules: (1) ESI methods submodule, which provide the unified interface for all the ESI methods to estimate the source activities; (2) Graphical model estimation submodule, which estimate the graphical model from the estimated source activities; and (3) The statistical comparison submodule, which compare the estimated graph model parameters at the same statistical significance level. In addition to all these submodules, EECoG_conComp also provides the ECoG Laplacian submodule, which estimates the Laplacian of ECoG over the cortical grid as another way of exploring the “pseudo source connectivity”. All these submodules will be detailed below.

With its wide acceptance, we shall use the ECoG Laplacian over the cortical grid as a surrogate measure for the primary current density \( {\varvec{J}} \). Note that, this estimator is very similar to the “spline Laplacian/dura images” discussed by Nunez and Srinivasan (2006).

The specific challenge of this submodule is to approximate the Laplacian over the same cortical grid that will be used for ESI methods. During this procedure, we are assuming that the ECoG Laplacian is smooth over the cortex. Suppose that \( {\varvec{V}}_{cortex} \) is the true voltage on the cortex for our cortical mesh, and the Laplacian on the cortex is \( \varvec\phi_{\;c} \). Thus, we have the voltage on the cortex (\( {\varvec{V}}_{cortex} = {\varvec{L}}^{\; - 1} \cdot {\varvec{\phi}}_{\;c} \)), where \( {\varvec{L}} \) is the Laplacian operator, and \( {\varvec{L}}^{\; - 1} \) is the inverse Laplacian operator. Then the signals measured by ECoG are written as:where \( {\varvec{S}} \) is a selection matrix that picks the voltage on the full cortical grid which are closest to the ECoG electrodes, and \( \varvec{\varepsilon}_{ecog} \) is the ECoG sensor noise with multi-variate Gaussian I.I.D. distribution. We emphasis that the ECoG Laplacian does not depend at all on the knowledge of the realistic head model or lead field. However, \( {\varvec{K}}_{Lap} = {\varvec{S}} \cdot {\varvec{L}}^{\; - 1} \) can be considered as a type of “pseudo lead field”, and analogues to most inverse problems, Eq. 3 is ill posed, and in order to estimate the Laplacian \( \varvec\phi_{\;c} \), we formulate this problem by penalizing the smoothness of the cortex Laplacian, then the problem can be solved by minimizing the following function:

This is a ridge type regression problem, which minimize the estimations square error with penalization of the 4th order Laplacian of the cortex voltage, or the 2nd order Laplacian on the cortex. In order to solve the problem defined in Eqs. 3 and 4, we substitute with \( {\varvec{X}} = {\varvec{S}} \cdot {\varvec{L}}^{\; - 2} \) and \( {\varvec{\varPhi}} = {\varvec{L}} \cdot \varvec\phi_{\;c} \). Then, Eq. 4 can be rewritten in the standard ridge regression form as:

After, minimizing this function and changing variables back, we obtain the estimator for the ECoG Laplacian on the cortical grid:

The hyper parameter \( \lambda \) is selected with generalized cross validation (GCV). The ECoG Laplacian estimator is calculated for each point of the cortical grid at each time instant, thus also yielding a “cortical grid time series” like the ESI method.

This submodule provides a uniform interface to the solution of the EECoG inverse problems and the estimators of the source current \( {\varvec{J}} \). At present, the inverse solutions implemented are listed in Table 1. These methods completely specified in the references given, and the METH toolbox provides the implementations as well. (https://​www.​uke.​de/​english/​departments-institutes/​institutes/​neurophysiology-and-pathophysiology/​research/​working-groups/​index.​html).

To compare the source connectivity, we need to ensure that all ESI methods reached their best performance. For MNE, LCMV and eLORETA, we use GCV to select the hyper parameters. For SSBL, the best performance is guaranteed by selecting the hyper parameters with its marginal likelihood stopping criteria.

Each type of inverse solution is calculated for each point on the cortical grid at each time instant. As mentioned before, we need to solve the inverse problem for both EEG and ECoG, then we can compare them on the same source space.

There are many proposed measures for brain connectivity, both directed and nondirected. We limit our attention of this platform to partial correlations as an undirected brain connectivity measure. This measure has been shown to be less sensitive to indirect connections (Dawson et al. 2016). They can be easily obtained from the inverse of the covariance matrix or precision matrix, which can be normalized to produce the partial correlation matrix (PCM). There are two specific challenges to be solved for this module:

We here take advantage of recently developed theory for inference of high dimensional sparse precision matrices from Jankova and van de Geer (2018) to address these challenges. The assumption of sparsity seems to be natural and widely adopted for estimators of brain connectivity and is relaxed with recent “debiasing” techniques explained below.

The input for the connectivity estimation submodule is the estimated time series at of the cortical grid from either ESI methods or the ECoG Laplacian. These time series are filtered to a specific frequency band of interest, like alpha of 8–12 Hz. From these filtered time series, we calculate the empirical covariance matrix. This empirical covariance matrix is then standardized to produce the empirical correlation matrix \( \hat{R} \). Then, both are used to obtain a sparse estimate of the partial correlations, also known as the weighted inverse covariance \(\hat{\Theta }_{w}\) which is estimated by the following minimizing function:where \( \left\| {\varvec{{\varTheta}}^{\; - } } \right\|_{1} \) is the \( {\varvec{l}}_{1} \) norm of the off-diagonal elements of the inverse covariance matrix \( {\varvec{\varTheta}} \), and \( \lambda \) is the hyper parameter. The computation is readily carried out by the QUIC algorithm (Hsieh et al. 2014), which is one optimized implementation of the graphical LASSO.

We leverage the recent theoretical results from Jankova and van de Geer (2018) in two ways:

First, we can correct the bias introduced by the graphical LASSO with the “debiasing” operation:

Second, asymptotic normality of the inverse covariances estimator is achieved as stated in Theorem 3 of Jankova’s paper (2018) with the following transformation:

This result allows us to find the thresholds for all the PCMs with the same statistical significance level.

Since we are dealing with very high dimensional PCMs, the threshold must be corrected for multiple comparisons, and it is done by the application of the positive False Discovery Rate \( pDFR \) (Storey 2002) as follows:

The connectivity comparison submodule offers mainly two methods to compares the thresholded estimated PCMs against a “base connectivity” matrix (either the truth known from simulations or any of the ESI based source connectivity estimators).

The first measure is based on ROC curves, which shows the true positive rate (TPR) against false positive rate (FPR) at all possible FPR values. In order to evaluate the significance of ROC curve, we also calculate the pointwise confidence bound for all the possible FPRs by bootstrap. We correct the bias introduced from the bootstrap distribution with bias-corrected and accelerated bootstrap interval (BCa interval) method, which is a second-order accurate interval that addresses these problems (Efron 1987; Huang et al. 2007). All the results reported in this paper have been confirmed by bootstrap permutation test (n = 200) with significance level of 0.05, which shows consistency with very small variances.

The ROC curves allow us to explore the effects of different pFDR choices on the false positive and false negative rates. ROC curves are usually summarized by the area under curve (AUC). Since we are only interested in low FDRs, we summarize the ROC curves with the partial area under the ROC curve.

The first metric to quantify the estimation performance is the standardized partial AUC (\( spAUC \)) (McClish 1989), which is independent of the level of the specified false positive rate (FPR). It is defined in the following equation, where \( e \) is the FPR level we are evaluating:

\( spAUC \) varies from 0 to 1, with 0.5 and 1 are the random and perfect diction. We use \( spAUC(0.1) \) in latter summary of results.

The second metric is the Performance Improvement Ratio (\( \text{PIR} \)), which compares the \( spAUC(e) \) of one method (M1) against another method (M2) with the same baseline. It is defined as follows:

PIR can be positive or negative. Positive \( {\text{PIR}} \) indicates that M1 outperforms M2, and vice versa.

Simulation is useful when we need to select the best available methods in a certain condition. For example, we have four ESI methods plus ECoG Laplacian, but we never know the “ground truth connectivity” beneath the real data or which is the most suitable method for our particular data. The only way to answer this question is to create a simulation as similar as possible to the experimental condition where the data is recorded. EECoG_scSim is designed to do so by simulating a structured partial correlation matrix as a simulated true source connectivity, and then generate the source activities with such connectivity. By using the EECoG lead fields generated from EECoG_FWM, we are able to simulate the same experiment with known source connectivity. Together with EECoG_conComp, EECoG-Comp provides tools for both testing methods and exploring data.

EECoG_scSim provides human based simulation for validation as well. A standard human forward model based on the MNI152 template (http://​www.​bic.​mni.​mcgill.​ca) and standard bioSemi electrodes layouts is calculated with BEM from BrainStorm (Tadel et al. 2011), and the corresponding lead field is provided for human simulation. EECoG_scSim allows the users to change the standard EEG electrode layouts from 10–20 system to 256 electrodes of full head coverage as well as the size of the cortical grid.

The theory behind this simulation is the same as the forward modeling procedure. The simulated sensor time series \( {\varvec{V}}_{LF} \) for a given head model \( LF \) is generated from:where \( {\varvec{J}}_{\varvec{{\varTheta }}} \) is the current source density sampled independently from distribution \( N({\varvec{0}},\;{\varvec{\varTheta}}^{ - 1} ) \), with \( {\varvec{\varTheta}} \) being the underlying PCM. \( {\varvec{K}}_{LF} \) is the lead field for a given head model. \( {\varvec{\varepsilon}}_{LF} \) is the sensor noise with \( {\varvec{\varepsilon}_{LF}} \sim N(\varvec{0},{\varvec{\xi}} \times {\varvec{I}}) \). The measurement signal to noise ratio (SNR) is 10 db for EEG and 20 db for ECoG. The length of each cortical grid time series can be the same as the real recordings.

Simulations of \( {\varvec{J}}_{\varTheta } \) are based on two different structured PCMs:

The lead fields used in the simulation module are \( {{\varvec{K}}_{H,eeg}} \) (Human EEG), \( {{\varvec{K}}_{M,eeg}} \) (Monkey EEG), \( {{\varvec{K}}_{M,ecog}} \) (Monkey ECoG) and \( {{\varvec{K}}_{Lap}}\) (Monkey ECoG Laplacian “pseudo lead field”).

We first evaluate the performance of all these methods in EECoG_scSim for both standard human EEG and monkey EECoG by means of simulations (with different number of sources, sensors and partial correlation matrix (PCM) structures); Then, we choose the best method from the monkey EECoG simulation to compared against; At last, we use all these ESI methods in exploring the source connectivity of the real data and give the final ranking of all these tested ESI methods as results.

These simulations are started with the generation of structured PCMs of the neural sources (or generators on the cortical mesh) denoted as \( {\varvec{\varTheta}} \), and they can have random and block structure which is specified as \( {\varvec{\varTheta}}_{rand} \) and \( {\varvec{\varTheta}}_{block} \) (more descriptions in Sect. 2.5). For simplicity, all symbols without specific suffices refer to general definition rather than a specific case—suffixes will be added only when necessary. Based on these two types of PCMs, the source activities \( {\varvec{J}} \) can be generated as \( {\varvec{J}}_{{{\varvec{\varTheta}}_{rand} }} \) and \( {\varvec{J}}_{{{\varvec{\varTheta}}_{block} }} \) without source noise. Both \( {\varvec{\varTheta}} \) and \( {\varvec{J}} \) are the same for all human and monkey simulations as the simulated “truth”. The different simulation settings are distinguished by the first term of suffices, \( HS \) for Human Simulation and \( MS \) for Monkey Simulation. We latter use \( MD \) to refer to real Monkey Data analysis. The sensor signals \( {\varvec{V}} \) are generated by multiplying the corresponding lead field \( {\varvec{K}} \) with the source activities \( {\varvec{J}} \) (corresponding to a given PCM \( {\varvec{\varTheta}} \)) with sensor noises \( {\varvec{\varepsilon}} \) added, this procedure can be described with the following equations:

\({\varvec{V}_{HS,EEG}}\) is the simulated human EEG with dimension \( N_{HS,chan} \times N_{tSample} \), \( N_{HS,chan} \in (19,32,64,128) \) is the number of human EEG channels, and \( N_{tSample} = 10800 \) is the number of time samples. The sampling frequency is 400 Hz for all simulations, which is kept the same as that in the preprocessed monkey recordings. \( {\varvec{K}_{HS,EEG}}\) is the Human EEG lead field with dimension \( N_{HS,chan} \times N_{HS,sour} \), \( N_{HS,chan} \) is the number of human EEG sensors and \( N_{HS,sour} \in (150,200,300,350) \). \( \varepsilon_{HS,EEG} \) is the human EEG sensor noise which follows I.I.D. Gaussian distribution \( \varepsilon_{HS,EEG} \sim N(0,\xi \times I) \) with dimension of \( N_{HS,chan} \times N_{tSample} \). \( {\varvec{V}_{MS,EEG}} \) and \({\varvec{V}_{MS,ECoG}} \) are generated in the similar way but with monkey EEG lead field \( {\varvec{K}}_{MS,EEG} \) (\( N_{MS,EEG} \times N_{MS,sour} \)) and monkey ECoG lead field \( K_{MS,ECoG} \) (\( N_{MS,ECoG} \times N_{MS,sour} \)). In order to select the best method to use on the actual monkey EECoG data (\( {\varvec{V}}_{MD,EEG} \) and \( {\varvec{V}}_{MD,ECoG} \)), we use the same number of the available monkey EEG channels \( N_{MS,EEG} = 15 \) and ECoG channels \( N_{MS,ECoG} = 125 \), as in the simulation, but changing the number of sources \( N_{MS,sour} \in (150,200,300,350) \), note that, \( N_{MS,ECoG} < N_{MS,sour} \) and \( N_{MS,EEG} < N_{MS,sour} \) always hold.

Then, we apply the four well-known ESI methods, namely MNE, eLORETA, LCMV and SSBL in human simulation (EEG), and MNE, eLORETA, LCMV and ECoG Laplacian (this last is viewed as another “pseudo inverse method”) in monkey simulations to reconstruct the source activities. The source activity estimator can be denoted as \({\varvec{\hat{J}_{{HS,EEG,\varTheta}}}}_{rand,ESI}\) for example, among all the sun-notations: (1) \( HS \) stands for human simulation, which can also be \( MS \) for monkey simulation and \( MD \) for monkey data, this is a notation for experimental condition; (2) \( EEG \) is the modality, and it can be \( ECoG \) as well; (3) \( {\varvec{\varTheta}_{rand}}\) is the underlying true random source connectivity matrix, which can alternatively be a blocked structured source connectivity matrix \({\varvec{\varTheta}_{block}} \), and it is unknown in the real data analysis; (4) \( ESI \) stands for different ESI methods, like MNE, eLORETA, LCMV, SSBL and ECoG Laplacian—specified when necessary.

With true source activities in theory J and estimated \( {\varvec{\hat{J}}}\), we are able to further estimate the underlying source level connectivity in terms of PCM. We take an example of \({\varvec{\hat{\varTheta}}}_{{HS,EEG,{\hat{\varvec{J}}_{{{\varvec{HS,EEG,\varTheta}}_{rand}}},ESI}}} \) to describe our notations for the connectivity estimators: (1) The first suffix indicates the type of experimental condition, which can be \( HS \) for Human Simulation, \( MS \) for Monkey Simulation and \( MD \) for Monkey Data analysis; (2) The second suffix is modality, which can be EEG or ECoG; (3) The third suffix is the source activities based on which this source connectivity matrix is estimated, for example it can simply be \( {\varvec{J_{{\varTheta_{rand}}}}} \), which means the direct estimation from the true source activities with underlying connectivity \( {\varvec{\varTheta}}_{rand} \), or like \({\varvec{\hat{J}_{{HS,EEG,\varTheta}}}}_{rand,ESI}\) in this example, which is the estimator from the reconstructed source activities obtained with a specific ESI method \( ESI \) for human EEG simulation (\( HS \)) with the underlying true source connectivity \( {\varvec{\varTheta}_{rand}} \).

We clarify that with this simulation we produce the true source activities \( {\varvec{J_{\varTheta}}} \) (no volume conduction) and its associated underlying true source connectivity \( {\varvec{\varTheta}} \). Before further analysis, we bandpass all the true source activities from 8–12 Hz to limit the connectivity estimations to alpha band. We then proceed to use the graphical LASSO (Hsieh et al. 2014) to find the “estimated true source connectivity” \({\varvec{\hat{\varTheta }_{{J_{\varTheta } }}}} \) which will help us quantify the error due to the partial correlation estimation procedure, without additional errors due to the source reconstruction.

With this machinery in place, we can generate simulated recordings of human EEG, monkey EEG and monkey ECoG based on models with different number of sources and different number of sensors. After band pass filtering, we apply different ESI methods to determine the “estimated source activities” \( \hat{J}_{\varTheta ,ESI} \) and finally apply the graphical LASSO to estimate the “ESI source connectivity” \( {\varvec{\hat{\varTheta }_{{\hat{J}_{\varTheta ,ESI} }}}} \).

To summarize the ESI methods performances in simulation, where we know the true source activities and connectivity, we use Performance Improvement Ratio (\( \text{PIR}_{ESI} \)) for a specific ESI method as a metric. For this case, M1 is the ESI estimated source connectivity and M2 is the estimated true source connectivity. For the analysis of the real MDR data, where we do not have true source activities and connectivity, we only report \( spAUC(0.1)_{ESI} \), which compares the ESI estimated source connectivity against the best ESI source connectivity estimator from the monkey simulations.

All these metrics are compared in the same experimental condition (like human simulation \( HS \)) with the same modality (like EEG), thus we omit these suffices here. To be specific in this simulation, \( \text{PIR}_{ESI} \) is defined as the accuracy improvement ratio of source PCM estimation (\( {\varvec{\hat{\varTheta }}_{{\varvec{{{\hat{J}}}}_{\varTheta ,ESI}}}} \)) from the reconstructed source activities from a specific ESI (\( {\varvec{\hat{J}}}_{\varTheta ,ESI} \)) against the direct true source PCM estimation (\( {\varvec{\hat{\varTheta }}_{{{\varvec{J}}_{\varTheta } }}} \)) from the true source activities (\( {\varvec{J}}_{\varTheta } \)). The accuracy is measured by \( spAUC(0.1) \) (see Sect. 2.4 for more details) and they are written as \( spAUC(0.1)_{ESI} \) and \( spAUC(0.1)_{J} \) respectively, which are calculated for both \( {\varvec{\hat{\varTheta }}_{{\varvec{{{\hat{J}}}}_{\varTheta ,ESI}}}} \) and \( {\varvec{\hat{\varTheta }}_{{{\varvec{J}}_{\varTheta } }}} \) by contrasting them with the true source connectivity matrix (\( {\varvec{\varTheta}} \)). \( \text{PIR}_{ESI} \) is calculated as follows:

Positive \( \text{PIR}_{ESI} \) indicates that this particular \( ESI \) method outperforms the direct estimation from the source activities, and vice versa. The whole simulation procedure is illustrated in the flowchart below in Fig. 6, and the results will be present in next section.

In the human simulation, we are testing the performance of different ESI methods on the standard human head model (MNI152 template http://​www.​bic.​mni.​mcgill.​ca) with different electrode layouts and different sizes of cortical grids. In this simulation, the number of sensors is \( N_{chan} \in (19,32,64,128) \) for 10–20 system, 32,64, and 128 channel system with whole head coverage, the number of sources is \( N_{sour} \in (150,200,300,350) \) over the whole cortex. Due to the computational load of graphical model estimation and ROC evaluation (bootstrapping for ROC confidence band), we are only simulating small cortical grids, therefore we have 16 results in total (four number of sensors times four grid sizes) for each type of source connectivity setting.

The results showed the ESI methods performance improvements in terms of \( \text{PIR}_{ESI} \) for both random and block sources for various sensor and source configurations. In summary, \( \text{PIR}_{ESI} \) of all the ESI methods improved less than 10%. The best performance was always achieved when the source connectivity had a block structure.

EECoG-Comp platform allows us to simulate simultaneous monkey EECoG data to test not only the ESI methods but also ECoG Laplacian described in Sect. 2.4. Due to the fixed number of EECoG channels in the real experiment, we only tested the performance of ESI methods across different sizes of cortical grids: \( N_{sour} \in (150,\;200,\;300,\;350) \), the methods tested in this simulation are MNE, LCMV, eLORETA and ECoG Laplacian. This means for each data (EEG or ECoG), we have four groups of results for each type of structured source connectivity.

The results from this simulation showed that the ECoG Laplacian almost always had the largest improvement, and the only exceptions were EEG and ECoG with 150 sources. In total, the performance differences between the ESI methods and the Laplacian are within 5%. Though ECoG Laplacian improves the source connectivity estimation the most, the results were not impressive and might not have practical implications. Like the human simulation, the best performance was always achieved when the source connectivity had a block structure.

Even though the ECoG Laplacian connectivity estimation did not yield huge performance improvement in the simulations, it did provide the highest \( \text{PIR}_{ESI} \). Therefore, we choose the ECoG Laplacian as a yardstick (even though we know it is an imperfect “ground truth”) when exploring the real data and compare the ESI methods against ECoG Laplacian.

In the MDR data analysis, we first cleaned the data with EECoG_Preproc, then input the data to EECoG_conComp. By applying different ESI methods, we estimated the source activities and ECoG Laplacian, band passed to alpha band (8–12 Hz), estimated the graphical model parameters with graphical LASSO and do the statistical evaluation of the similarity between the ESI reconstructed partial correlation matrix (PCM) and ECoG Laplacian PCM. The results are summarized in the bar diagram below in Fig. 7 in terms of \( spAUC(0.1) \). Note that it is a different metric from \( \text{PIR}_{ESI} \), the one we use to compare different ESI methods in. All \( spAUC(0.1) \) values are within the range of [0.61–0.74] with mean 0.65 and standard deviation 0.05.

It is also interesting to see that all the methods are giving “higher \( spAUC \) for the awake state data, and ECoG outperforms EEG more than 10%. On the contrary, in anesthesia state, all the methods give equally low accuracy compared with ECoG Laplacian, there is even no clear performance difference between EEG and ECoG in the anesthesia state data.