A convolution and attention-based conditional adversarial domain adaptation neural network for emotion recognition using electroencephalography

Electroencephalograph (EEG)-based emotion recognition is a crucial task in affective computing and human–computer interaction. The development of deep learning techniques has permitted significant improvement in emotion recognition tasks. For instance, convolutional neural networks and recurrent neural networks are proficient in capturing spatial information from electrodes and temporal information from EEG signals, respectively. However, most of the previous studies relied solely on stacking such models, and only utilized portion of features from different domain. In addition, most models ignored the value of information transfer between experimental samples that may be critical to advancing the performance of cross-subject emotion recognition task. To overcome these issues, we proposed a convolution and attention-based conditional adversarial domain adaptation neural network (CA²DANet) for emotion recognition tasks. The model consisted of a differential entropy feature extractor, a feature fusion network, a class predictor and a domain discriminator, permitted to extract deep features of different domain from EEG signals. For evaluation, we conducted experiments on the SEED dataset, which contains 62-channel EEG data acquired during three types of emotions: positive, neutral and negative. The experimental results on the benchmark dataset showed a promising improvement of classification performance compared with the different methods with accuracies of 96.29% and 81.05% respectively on subject-dependent and subject-independent experiments. Additionally, we assessed the crucial frequency bands and EEG channels automatically selected by CA²DANet and, through a t-SNE visualization, to examine the efficacy of conditional adversarial domain adaptation. Overall, our study indicates the propose CA²DANet is a promising model for EEG emotion recognition tasks.