一种面向运动解码的 EEG-fNIRS 时频特征融合 与协同分类方法

脑功能成像技术可以反映人体运动时的大脑生理变化,进而解码运动状态,但单模态信号反映的大脑生理信息存在局 限性。 为此,本文提出了一种基于 EEG 和 fNIRS 信号的时频特征融合与协同分类方法,利用脑神经电活动和血氧信息的互补 特性提高运动状态解码精度。 首先,提取 EEG 的小波包能量熵特征,使用双向长短期记忆网络(Bi-LSTM)提取 fNIRS 的时域特 征,将两类特征组合得到包含时频域信息的融合特征,实现 EEG 和 fNIRS 不同层次特征的信息互补。 然后,利用 1DCNN 提取 融合特征深层次信息。 最后,采用全连接神经网络进行任务分类。 将所提方法应用于公开数据集,本文所提的 EEG-fNIRS 信号 协同分类方法准确率为 95. 31% ,较单模态分类高 7. 81% ~ 9. 60% 。 结果表明,该方法充分融合了两互补信号的时频域信息,提 高了对左右手握力运动的分类准确率。

A time-frequency feature fusion and collaborative classification method for motion decoding with EEG-fNIRS signals

Functional neural imaging technology can reflect the physiological change of the brain, and decode the movement state. However, the information by the single neural imaging modality is limited. In this article, a time-frequency feature fusion and collaborative classification method is proposed to achieve high precision motion state decoding with EEG and fNIRS signals, which takes the advantage of the complementation of electrical activity and hemoglobin changes. Firstly, the wavelet packet energy entropy feature of the EEG signal is extracted, the Bi-LSTM deep neural network is used to extract the time domain features of the fNIRS signal, and the achieved features are combined to obtain the fusion features containing the time-frequency domain information. The complementation of EEG and fNIRS features is achieved at multiple levels. Then, the 1DCNN is used to extract deep-level information from the fusion features. Finally, a fully connected neural network is used for classification. The proposed method has been tested with a public dataset. The EEG-fNIRS collaborative classification method achieves the accuracy of 95. 31% , which is 7. 81% ~ 9. 60% higher than those of single-modal signal classification methods. Experimental results show that this method fully integrates the timefrequency domain information of two physiologically complementary signals, and improves the classification accuracy of left and right hand grip tasks.