Towards adaptive brain-computer interfaces: statistical inference for mental state recognition

Official URL: https://risc01.sabanciuniv.edu/record=b2486382 _(Table of contents)

Brain-computer interface (BCI) systems aim to establish direct communication channels between the brain and external devices. The primary motivation is to enable patients with limited or no muscular control, including amyotrophic lateral sclerosis (ALS) and stroke patients, to use computers or other devices by automatically interpreting their intent based on the measured brain electrical activity. Furthermore, enabling healthy individuals to use BCI systems as an additional communication channel in certain human computer interaction systems is also a current topic of interest. Current experimental BCI systems are trained in a supervised fashion and then evaluated during test sessions. With increasing demands for daily and long-term use of BCIs in real-life applications such as in semi-autonomous cars, BCIs have been tested on longer sessions in which researchers have observed considerably lower performance of trained systems. This is believed to be caused by the nonstationary nature of the electroencephalographic (EEG) signals. As a result, semi-supervised adaptation of BCI systems based on test data has emerged as a new research domain. One of the main reasons underlying the nonstationarity of signals involves changes in the users’ cognitive states such as the cognitive load, alertness, attention, fatigue, boredom, and motivation. However, dynamically extracting information about such cognitive states from EEG signals and using that to improve the performance of BCI systems is currently an open research problem. In this thesis, we tackle the highly complex problem of estimating the level of alertness and vigilance of users during execution of cognitive tasks. To identify the neural, EEGbased correlates of long-term task and response time consistency, we devise a series of experiments running the sustained attention to response task (SART). After proposing a novel adaptive scoring scheme for vigilance, we provide new evidence on the close relationship between intrinsic resting and task-related brain networks and develop models to predict consistency in tonic performance and response time using neural networks and feature relevance analysis from spatio-spectral features of resting-state EEG signals. Next, focusing on the imminent goal of predicting low and high vigilance intervals, we propose fully automated systems based on convolutional neural networks (CNNs) using phase locking value features as successful pre-trial predictors of phasic vigilance and performance consistency. In all of these contributions, we consider the personal vigilance traits and individual psychophysiological differences for modeling and detecting the extremely alert and drowsy trials in long and monotonous experiments, and enrich the literature with the evidence on spatio-spectro-temporal correlates of vigilant and consistent behavior. We then utilize Bayesian changepoint models for sequential inference and detection of instants at which continuous vigilance levels of users enter a new phase. We demonstrate the success of our online and offline vigilance models in detecting changepoints from both the SART datasets collected in our lab and driving datasets that contain vigilance labels. Finally and as the highlight of this thesis, we hypothesize that the underlying vigilance levels affect users’ reaction time and thus the ability to focus and engage in motor imagery BCI paradigms. We then introduce an adaptive alertness-aware MI classification system for motor imagery BCI that uses a series of novel unsupervised learning schemes for labeling trial vigilance levels during training and test sessions, and leads to a method with full adaptation in both feature extraction and training of its classifier parameters. Three different versions of this adaptive classification approach are introduced that are trained differently on trials labeled with low vigilance levels by our various vigilance clustering schemes. We report improvements in the overall test accuracy of adaptive versions with respect to the original, non-adaptive baseline for our own SPIS MI-BCI dataset and the BCI Competition IV Dataset 2a. A number of datasets collected in our BCI laboratory are uploaded to a public repository at https://github.com/mastaneht.