Gaze-based attention refocusing training in virtual reality for adult attention-deficit/hyperactivity disorder

The study was advertised via the adult ADHD specialist outpatient clinic of the Department of Psychiatry and Psychotherapy of the University Hospital Bonn and via publicly accessible media. Of 40 participants who entered the study between February 2021 and August 2021, 36 completed the participation (for the participant flow, see Fig. 2). To be eligible, participants had to be between 18 and 65 years of age, have normal or corrected-to-normal vision, adequately understand the study content and language, not be pregnant, not have epilepsy, not have oculomotor atypicalities, and not have rashes on the scalp. Moreover, all participants assigned to the ADHD group had to meet the DSM-5 diagnostic criteria for ADHD as assessed with the revised, German version of the Clinical Interview for the Integrated Diagnosis of ADHD in Adulthood (IDA-R) [3, 47]. Additionally, they had to be free of a schizophrenia spectrum disorder, severe affective disorder, antisocial personality disorder, or moderate-to-severe substance abuse. Also, participants had to discontinue taking their ADHD medication 48 h before the experiment. Healthy participants, in turn, were ineligible if they had a psychiatric diagnosis as mentioned above or a diagnosis of ADHD. Therefore, before study participation, all potential participants were screened with the Brief Diagnostic Interview for Mental Disorders (Mini-Dips-OA, German version) [48] and the Assessment of DSM-IV Personality Disorders (ADP-IV, German version) [49].

The study was conducted in accordance with the Helsinki Declaration as revised in 2013, and approved by the local medical ethics committee of the University of Bonn (protocol number: 297/20). A required sample size of 36 participants was revealed by an a priori sample size calculation in G*Power [50], based on an alpha error probability of 0.05, a power of 0.9 and a moderate effect size. Written informed consent was obtained from all participants. Information that could identify participants is not published. For compensation, participants had the opportunity to enter a draw (2 × 50 €). The trial was preregistered in the German WHO primary registry DRKS on 01–12-2020 (identifier: DRKS00022370).

The study was designed as a two-armed controlled trial, in which two groups (patients with ADHD, HC) received three feedback conditions in counterbalanced order: real feedback, sham feedback, and no feedback. The real feedback condition thereby served as the main intervention of interest, during which audiovisual feedback was triggered as soon as an eye-tracked gaze behavior was registered that indicated a loss of task focus (details below). The sham feedback and no feedback conditions, in turn, served as control conditions. Whereas in the sham feedback condition, the same type of audiovisual feedback was provided as in the real feedback condition, except that here the feedback was provided with a quasi-random delay (20—30 s) after inattention registration, the no feedback condition provided no feedback at all. Otherwise, all three conditions were identically structured: Participants were immersed into the VSR, i.e., a virtual testing environment of high ecological validity for the multimodal assessment of ADHD-associated symptoms, and performed a CPT while distracting events occurred. Participants were blind to which CPT block represented which condition, but were informed that feedback could appear at any time and in any condition.

Study participation was scheduled over two days: Day 1 served for the eligibility testing and clinical characterization of our participants and was conducted online as a result of COVID-19 restrictions for some participants. Day 2 included the experiment and occurred at the virtual reality laboratory of the Department of Psychiatry and Psychotherapy of the University Hospital Bonn. The total duration was approximately 1.5 h and 2.5 h for Day 1 and 2, respectively.

ADHD symptoms were evaluated based on both the observer-rated clinical interview IDA-R [47] and the self-rating behavior questionnaire ADHS-SB [51]. Moreover, the World Health Organization Quality Of Life questionnaire (WHOQOL) [52] and the Depression, Anxiety and Stress Scales (DASS) [53] were completed for further clinical characterization. Demographic data were collected with a lab-internal questionnaire.

The experimental procedure on Day 2 was as follows: First, participants were prepared for the EEG recordings, before they were seated at a 1 × 1 m table with a keyboard in front of them. Next, the head-mounted display (HMD) was placed on the participants' heads. The HMD used was the HTC Vive Pro Eye (HTC Corporation, Taoyuan City, Taiwan), which has 1440 × 1600 pixels per eye image resolution, a 90 Hz screen refresh rate, a 110-degree field of view and an embedded eye-tracking system. Immersed into the VSR, participants found themselves seated at a virtual table from where they could follow the VSR scenery from a first-person perspective (cf. Fig. 1). Besides the canvas, which was located at the front of the VSR and on which the CPT was presented, typical seminar room equipment and animated study mates were included. The VSR has been self-assembled by our lab using Unity 3D 2019.1.10f1 (Unity Technologies, San Francisco, CA, USA) and C# based on pre-existing assets (e.g., 3D Everything's School Classroom which is available in the Unity Asset Store). Its complete functionality and validation has previously been described in detail [46]. After the participants had briefly accustomed to the virtual environment, a short calibration sequence for the eye-tracking system followed, before a first trial run of the ensuing CPT task was conducted. Next, the three feedback conditions were run, with two-minute breaks and recalibrations of the eye-tracker between each condition. All three conditions consisted of performing a CPT for 18 min (cf. section continuous performance task), while additional distractor events occurred (cf. section implementation of distracting events) and, if applicable, audiovisual feedback was given. Each condition ended with an experience sampling, in which the participants were briefly surveyed about their subjective experiences via a VR-embedded survey tool. In addition, after all CPT blocks were completed, the Virtual Reality Sickness Questionnaire (VRSQ) [54] was presented, before participants removed the HMD. In total, participants remained in the virtual environment for about one hour. Finally, participants completed a recognition test regarding perceived distractor stimuli during the virtual experiment via a desktop screen.

The CPT was directly presented on a canvas at the front wall of the VSR (cf. Fig. 1). Specifically, a sequence of single letters was presented centrally and iteratively on the canvas, with a stimulus duration of 100 ms and an interstimulus interval of 1100 ms, resulting in 900 trials per block. The task required pressing a key as quickly as possible when a "K" was shown after an "A", while withholding the response for any other sequence of letters. Compared to our previous VSR study [46], in which we found a ceiling effect (i.e. a very low error rate), a faster stimulus sequence was applied by decreasing the interstimulus interval by 800 ms. In each CPT block (i.e. each condition), 30% target sequences and 70% non-target sequences were presented. Of the latter, 50% were pseudo-targets containing only one of the two target letters. For analysis, reaction times (in ms) of all responses, commission errors (as an estimation of impulsivity) and omission errors (as an estimation of inattention) were derived.

Each CPT block (i.e., condition) was further divided into alternating distractor phases (DP) and non-distractor phases (NDP), with each of these phases lasting three minutes. Whether a CPT block started with a DP or NDP was counterbalanced across participants. During an NDP, the seminar room was presented unchanged. During each DP, a total of six different visual, auditory, and audiovisual distractors were randomly selected (from a pool of 18 visual, 18 auditory, 18 combined audiovisual distractors) and presented in intervals of 30 s. The distractors represented events with high everyday relevance, such as a smartphone ringing or birds flying past the window and were widely balanced (28:26) in terms of their content reflecting a social (e.g., a person entering the room) or non-social (e.g., a passing fire truck) context.

Eye movements were recorded with a sampling frequency of ~ 50 Hz via the infrared-based Tobii eye-tracker (Tobii Technology, Stockholm, Sweden) built into the HMD. The eye-tracker has an accuracy estimation of 0.5°—1.1° and allows the additional wearing of glasses, which was required in 39% of patients and 11% of healthy participants. Participants were asked not to wear any eye makeup. Eye-tracking data were recorded by a combination of three different software packages: SRanipal SDK version 1.3.1.1 (HTC Corporation, Taoyuan, Taiwan), Tobii XR SDK version 1.16.36.0 (Tobii Technology, Stockholm, Sweden), and Lab Streaming Layer (LSL; https://​github.​com/​sccn/​labstreaminglaye​r). SRanipal SDK provided access to the raw eye-tracking data within Unity. Tobii XR SDK was used to track the participant’s momentary gaze focus on specific virtual objects within Unity. Technically, this tracking was realized by the SDK’s IGazeFocusable interface that builds upon Unity’s collider system and allows to register whenever a specified collider (3D object) is hit by a raycast representing the participant’s gaze direction. Using this functionality, three different eye gaze states were defined and tracked:

For each recorded time stamp, only one of the three possible gaze direction states (excluding blinks) was thereby possible at a time. Finally, LSL was used to save the eye-tracking data along with the other data collected.

As stated, during both the real feedback and sham feedback conditions, audiovisual feedback was triggered whenever gaze locations indicated a loss of task focus. A loss of task focus was assumed as soon as a participant did not look at the canvas for more than 2 s or as soon as a participant gazed at a distractor for more than 0.5 s. In the real feedback condition, this resulted in an immediate provision of audiovisual feedback. In the sham feedback condition, an initial delay of 20—30 s was implemented before feedback initiation to ensure a similar frequency compared to the real feedback. The audiovisual feedback itself consisted of a 0.5 s black fade-in effect to a maximum of approximately 35% of the screen size combined with a chime-like sound effect (for a video presentation, see Supplementary Material 1). It was automatically stopped as soon as either the gaze was redirected to the canvas or 2 s passed. In addition, following feedback, there was a refractory period of 5 s during which no further feedback could be played to prevent over-extensive initiation of feedback.

Eye-tracking offline analyses were conducted in Matlab 2021b (The MathWorks Inc., Natick, MA, USA). Detection of saccades and fixations was based on a custom Matlab script that implemented an adaptive data-driven algorithm for velocity-based detection (for details, see [55]). Specifically, the three-dimensional gaze coordinates of each eye were used to calculate sample-to-sample velocities and accelerations [56]. A second order Savitzky-Golay finite impulse response filter was applied for data smoothing [57]. Invalid data as indicated by the SRanipal validity score were discarded from analysis. Interpolation across gaps of 75 ms maximum duration was performed linearly. Loss of data from one side was compensated using valid data from the other eye, and subsequently the data from both eyes were averaged. Implicitly detected fixations with a duration of less than 60 ms were discarded and fixations were merged on the basis of inter-fixation intervals of maximum 40 ms. Mean data loss was 2.75% (SD = 1.98%) per participant. For analysis, the average number of saccades and average saccade durations (in ms) were derived for each condition and phase.

The analysis of the gaze direction behavior, in turn, focused on the three gaze direction states, which were already online determined in Unity during the experiment and tracked by LSL. For statistical analyses, the following dwell times were separately derived for each group and each feedback condition and, additionally, a composite distractibility score was calculated:

A high distractibility score thereby indicates a high level of distraction.

The EEG was gathered via a wireless EEG system (Smarting®, mBrainTrain®, Belgrade, Serbia) and electrodes were placed by means of an EEG cap (EASYCAP, Herrsching, Germany) according to the 10–20 system and included 24 Ag/AgCl sintered ring electrodes: Fp1, Fp2, AFz, F3, Fz, F4, T7, C3, Cz, C4, T8, CPz, P7, P3, Pz, P4, P8, POz, O1, O2, M1, and M2, with the ground electrode (DRL) at FPz and the reference electrode (CMS) at FCz. With impedances kept < 10 kΩ, EEG data was digitized via LSL at a 500 Hz sampling rate and a 24-bit step-size resolution.

For the offline analysis, Matlab 2021b and EEGLAB 2021.0 [58] were used. First, the EEG datasets were temporally filtered between 0 and 35 Hz, detrended, and subsequently screened for noise in EEG channels. In each of 4 datasets, one noisy EEG channel was identified and replaced via spherical interpolation. Next, for calculating an independent component analysis (ICA), the continuous EEG data was epoched into 2 s time windows and non-stereotypic artifacts were removed using built-in EEGLAB functions. After that, an ICA was computed and components containing stereotypical artifacts such as ocular, cardiac, or muscle activity, were visually identified, backprojected to the continuous EEG data, and then rejected. The visual inspection of the components was thereby conducted by a trained EEG researcher and based on built-in functions of EEGLAB and focused on the components scalp topographies, spectral characteristics and time courses. The ICA-corrected continuous EEG datasets were cut into six separate subsets (either all DP or NDP within one feedback condition). Subsequently, every subset was epoched into as many non-overlapping five-seconds segments as possible, these segments were baseline corrected and all segments containing nonstereotyped artifacts were rejected. A continuous wavelet transformation was calculated on each retained segment for channel Fz. The time resolution amounted to 4 ms and the frequency range ranged from 0.1 to 35.0 Hz in 85 steps on a log scale. Finally, the average theta (4—7 Hz) and beta (13—30 Hz) power across segments was calculated between 0.5 and 4.5 s and the TBR was derived by dividing the theta power values by the beta power values.

Head movement as a measure of actigraphy was obtained from built-in positional tracking of the HTC Vive system. The Euclidean 3D coordinates were recorded via LSL with a ~ 90 Hz sampling rate. For offline analysis in Matlab 2021b, the raw data was first downsampled to ~ 10 Hz and then the Euclidean distances between each consecutive 3D position of the HMD were computed. Finally, the mean distances of head position shifts were obtained.

After each feedback condition, a gesture-controlled user interface was provided to assess the participant’s momentary ADHD core symptoms. The user interface showed up as a semi-transparent overlay directly within the VSR and evaluated the participant’s symptoms of inattention, hyperactivity, and impulsivity on a 7-point Likert scale from -3 (no symptoms) to 3 (serious symptoms). Also, satisfaction with the GART and cybersickness via the VRSQ were inquired via this user interface.

After completion of the experiment, a recognition task was administered in which 60 visual or auditory distractors were shown. Of these, 50% represented actual distractors played during the experiment and 50% represented distractors that were unplayed. Upon each presented distractor, participants had to decide whether this distractor was encountered during the experiment, or not. The recognition accuracy, i.e., the proportion of correct responses out of all possible correct responses, was derived for analysis.

Complete data sets were available for all variables except the recognition task, which was not completed by two participants. The corresponding analysis of the recognition task was based on the remaining complete data sets.

With regard to ANOVA assumptions, visual inspection of Q-Q plots and histograms indicated non-normally distributed data in some cases. However, no serious violations were detected, and given the robustness of ANOVAs to non-normality [59], analyses were continued as planned. Sphericity violations were adjusted with the Huynh field correction for ε > 0.75 and Greenhouse–Geisser corrections in the remaining cases (see Supplementary Material 2).

To investigate potential differences between groups, feedback conditions, and phases, separate 2 × 3 × 2 mixed ANOVAs with the between-subjects factor Group (ADHD vs. HC) and the within-subject factors Feedback Condition (real feedback vs. sham feedback vs. no feedback) and Phase (DP vs. NDP) were carried out on commission errors, omission errors, reaction times, saccade durations, number of saccades, TBR values, and head movements. Moreover, 2 × 3 mixed ANOVAs (Group × Feedback condition) were conducted on gaze dwell time percentages (task focus, distractor focus, and gaze wandering), the composite distractibility score, and on the separate ADHD core symptom outcome scores of the experience sampling (inattention, hyperactivity, and impulsivity). Post-hoc comparisons were based on Bonferroni-adjusted t-tests. Independent samples t-tests were conducted to assess group differences with respect to VR-related cybersickness and satisfaction with the GART. Additionally, one-sample t-tests were carried out against "0" (moderate cybersickness/satisfaction) to determine differences from neutral responses. The accuracies of the recognition task were compared between both groups by using an independent samples t-test.

Finally, for an investigation of potential associations between measures, Pearson and Spearman's rank correlations with Benjamini–Hochberg corrected p-values were calculated separately for each group and feedback condition for several outcome parameters [60]. These included all previously described CPT and eye movement parameters, EEG theta power and beta power, head movements, the number of feedback triggered (except for the no feedback condition), and the ADHD total symptom scores as measured by experience sampling, the IDA-R, and the ADHS-SB. Age and education were the only demographic parameters evaluated.

All statistical tests were performed two-sided with a significance level of α = 0.05. Due to the exploratory nature of this study, which, to our knowledge, is the first to implement such simultaneous recording of multimodal physiological and behavioral data streams in VR, and which is intended to act hypothesis-generating for future confirmatory trials, unadjusted p-values are presented (except for the correlation analyses) with respect to multiple testing [61, 62]. Analyses were performed in SPSS 21.0 (IBM Corp., Armonck, NY, USA), except for the correlation analyses, which were performed in Matlab 2021b (The MathWorks Inc., Natick, MA, USA) and R software 3.6.1 [63] and visualized by means of the Corrplot package for R version 0.84 [64].

Overall, 18 adult outpatients with ADHD (6 females) and 18 HC (7 females) participated in the present study. All of them were recruited in Germany and identified as of White European ethnicity. Detailed sample characteristics are presented in Table 1.

The results of the behavioral CPT performance are depicted in Fig. 3 (A—C). No significant main effect of Feedback Condition or interactions between Phase, Feedback Condition and Group were detected for reaction times, omission errors or commission errors.

For omission errors, a significant main effect of Phase (F(1,34) = 9.35, p = 0.004, ηp² = 0.22) was found, in that significantly more omission errors were made during DP (M = 2.02; 95% CI [1.11, 2.92]) compared to NDP (M = 1.67; 95% CI [0.90, 2.44]). Likewise, at least descriptively (F(1,34) = 3.74, p = 0.061, ηp² = 0.10) more commission errors were observed during DP (M = 1.19, 95% CI [0.58, 1.80]) than NDP (M = 1.00, 95% CI [0.51, 1.49]).

Patients with ADHD and HC differed in omission errors (F(1,34) = 5.57, p = 0.024, ηp² = 0.14) and reaction times (F(1,34) = 4.37, p = 0.044, ηp² = 0.11). Across phases and feedback conditions, the ADHD group committed more omission errors (M_OE = 2.81, 95% CI [1.63, 3.99]) and had slower reaction times (M_RT = 471.46 ms, 95% CI [446.85, 496.07]) than the HC group (M_OE = 0.88, 95% CI [-0.30, 2.05]; M_RT = 435.64 ms; 95% CI [411.03, 460.26]).

To evaluate the participants' gaze behavior during CPT performance, four gaze direction parameters were analyzed (cf. Fig. 3D - G): time of task focus, time of distractor focus, time of gaze wandering, and a composite distractibility score. For none of the four gaze direction parameters, a significant main effect of Feedback Condition or an interaction between Feedback Condition and Group was shown.

Instead, a significant group difference was found regarding the time of distractor focus (F(1,34) = 9.40, p = 0.004, ηp² = 0.22), in that patients with ADHD spent more time (M = 1.86%; 95% CI [1.38%, 2.34%]) gazing at distractors than HC (M = 0.84%; 95% CI [0.36%, 1.32%]). Comparing the time of attending the canvas between the ADHD group (M = 86.35%; 95% CI [82.14%, 90.56%]) and HC (M = 91.81%; 95% CI [87.60%, 96.01%]), healthy individuals showed only descriptively a higher percentage (F(1,34) = 3.48, p = 0.071, ηp² = 0.09). In line with these indications, there was also a trend for a higher distractibility composite score in patients compared with HC (F(1,34) = 3.68, p = 0.064, ηp² = 0.10).

For the average duration and the number of saccades, the ANOVAs revealed no interactions, but showed longer (F(1,34) = 11.73, p = 0.002, ηp² = 0.26) and a higher number of saccades (F(1,34) = 13.87, p = 0.001, ηp² = 0.29) during DP than NDP. In addition, we found a higher number of saccades (F(1,34) = 4.90, p = 0.034, ηp² = 0.13) but only descriptively longer saccade durations (F(1,34) = 3.90, p = 0.057, ηp² = 0.10) in ADHD than in HC.

EEG analyses focused on spectral differences concerning the participants' TBR. Time–frequency power spectra of the conducted wavelet analyses are presented in Fig. 4A and B, whereas the TBRs are depicted Fig. 4C. We found no significant main or interaction effects for Feedback Condition or Group, but a main effect of Phase (F(1,34) = 18.02, p < 0.001, ηp² = 0.35), in that the TBR was higher during DP (M = 1.07; 95% CI [0.97, 1.18]) than NDP (M = 1.05; 95% CI [0.95, 1.16]).

Actigraphy analyses focused on differences in head movements. While the ANOVA indicated a significant main effect of Feedback Condition (F(2,68) = 3.58, p = 0.033, ηp² = 0.10), Bonferroni-adjusted post-hoc comparisons only yielded a trend (p = 0.053) toward more head movements during sham feedback compared to no feedback (M_Diff = 0.20; 95% CI [-0.002, 0.40]). We further found a significant main effect of Group (F(1,34) = 16.06, p < 0.001, ηp² = 0.32), in that patients with ADHD exhibited more head movements (M = 1.75; 95% CI [1.41, 2.09]) than HC (M = 0.80; 95% CI [0.46, 1.14]).

To determine the subjective experience of momentary ADHD symptomatology, a short experience sampling was conducted after each CPT block, in which the participants rated their levels of inattention, impulsivity and hyperactivity. For none of the three parameters, any significant main effect of Feedback Condition or interaction effect emerged. Nonetheless, we found significant group differences for symptoms of inattention (F(1,34) = 19.57, p < 0.001, ηp² = 0.37), hyperactivity (F(1,34) = 16.96, p < 0.001, ηp² = 0.33), and impulsivity (F(1,34) = 8.76, p = 0.006, ηp² = 0.21), in that for all three ADHD symptoms higher ratings were observed in patients with ADHD than in HC.

The participants’ VR-related cybersickness was rated significantly lower than “0” (moderate sickness) by means of the VRSQ (t(35) = -2.76, p = 0.009, d = -0.46). Groups did not differ significantly (t(34) = 1.45, p = 0.156, d = 0.49) and, on average, scores of -0.25 (95% CI [-0.88, 0.38]) in the ADHD group and -0.78 (95% CI [-1.25, -0.32]) in the HC group were obtained. In line with this, no adverse events were reported with respect to the VR experiment.

Additionally, the participants’ satisfaction with the developed GART was reported to be significantly higher than “0” (moderate satisfaction) after the experiment (t(35) = 3.62, p = 0.001, d = 0.60). Patients with ADHD (M = 1.56, 95% CI [0.79, 2.32]) and HC (M = 0.61 (95% CI [-0.36, 1.58]) were similarly satisfied with their VR experience (t(34) = 1.61, p = 0.116, d = 0.55).

Of the 60 potential distractors shown to participants after the experiment as a recognition test, only 50% actually represented GART-implemented distractors. Patients with ADHD (M = 70.46%; 95% CI [65.76%, 75.17%]) and HC (M = 72.71%; 95% CI [68.96%, 76.46%]) classified the presented distractors with similar recognition accuracies (t(32) = -0.78, p = 0.443, d = -0.27).

Correlation matrices for several primary and secondary outcome parameters are depicted separately for groups and feedback conditions in Fig. 5. Correlation analyses across all feedback conditions revealed clusters of strong correlations within measurement domains (e.g. between time of task focus and time of gaze wandering). In ADHD but not in HC, saccade durations appeared to correlate positively with other physiological measures of inattention, such as CPT omission errors and gaze wandering, and negatively with times of task focus under various feedback conditions. Regarding EEG, the theta and beta power were positively correlated across feedback conditions and groups. Of note, participant age and education were included as the only demographic parameters and, besides a negative correlation between education and time of distractor focus in HC during sham feedback (Spearman’s rank correlation, r(34) = -0.76, p = 0.005) and of education and number of saccades during real feedback (r(34) = -0.68, p = 0.031), no significant correlations with any parameter presented were observed.