Modified mixture of experts employing eigenvector methods and Lyapunov exponents for analysis of electroencephalogram signals

Abstract: The use of diverse features in detecting variability of electroencephalogram (EEG) signals is presented. The classification accuracies of the modified mixture of experts (MME), which was trained on diverse features, were obtained. Eigenvector methods (Pisarenko, multiple signal classification – MUSIC, and minimum-norm) were selected to generate the power spectral density estimates. The features from the power spectral density estimates and Lyapunov exponents of the EEG signals were computed and statistical features were calculated to depict their distribution. The statistical features, which were used for obtaining the diverse features of the EEG signals, were then input into the implemented neural network models for training and testing purposes. The present study demonstrated that the MME trained on the diverse features achieved high accuracy rates (total classification accuracy of the MME is 98.33%).     

Analysis of EEG signals by implementing eigenvector methods/recurrent neural networks

The implementation of recurrent neural network (RNN) employing eigenvector methods is presented for classification of electroencephalogram (EEG) signals. In practical applications of pattern recognition, there are often diverse features extracted from raw data which needs recognizing. Because of the importance of making the right decision, the present work is carried out for searching better classification procedures for the EEG signals. Decision making was performed in two stages: feature extraction by eigenvector methods and classification using the classifiers trained on the extracted features. The aim of the study is classification of the EEG signals by the combination of eigenvector methods and the RNN. The present research demonstrated that the power levels of the power spectral density (PSD) estimates obtained by the eigenvector methods are the features which well represent the EEG signals and the RNN trained on these features achieved high classification accuracies.     

A modified mixture of experts network structure for ECG beats classification with diverse features

This paper illustrates the use of modified mixture of experts (MME) network structure to guide model selection for classification of electrocardiogram (ECG) beats with diverse features. The MME is a modular neural network architecture for supervised learning. Expectation-maximization (EM) algorithm was used for training the MME so that the learning process is decoupled in a manner that fits well with the modular structure. The wavelet coefficients and Lyapunov exponents of the ECG signals were computed and statistical features were calculated to depict their distribution. The statistical features, which were used for obtaining the diverse features of the ECG signals, were then input into the MME network structure for training and testing purposes. We explored the ability of designed and trained MME network structure, combined with wavelet preprocessing (computing wavelet coefficients) and nonlinear dynamics tools (computing Lyapunov exponents), to discriminate five types of ECG beats (normal beat, congestive heart failure beat, ventricular tachyarrhythmia beat, atrial fibrillation beat, partial epilepsy beat) obtained from the Physiobank database. The MME achieved accuracy rates which were higher than that of the mixture of experts (ME) and feedforward neural network models (multilayer perceptron neural network—MLPNN). The proposed MME approach can be useful in classifying long-term ECG signals for early detection of heart diseases/abnormalities.     

Combining eigenvector methods and support vector machines for detecting variability of Doppler ultrasound signals

In this paper, the multiclass support vector machines (SVMs) with the error correcting output codes (ECOC) were presented for detecting variabilities of the multiclass Doppler ultrasound signals. The ophthalmic arterial (OA) Doppler signals were recorded from healthy subjects, subjects suffering from OA stenosis, subjects suffering from ocular Behcet disease. The internal carotid arterial (ICA) Doppler signals were recorded from healthy subjects, subjects suffering from ICA stenosis, subjects suffering from ICA occlusion. Methods of combining multiple classifiers with diverse features are viewed as a general problem in various application areas of pattern recognition. Because of the importance of making the right decision, better classification procedures for Doppler ultrasound signals are searched. Decision making was performed in two stages: feature extraction by eigenvector methods and classification using the SVMs trained on the extracted features. The research demonstrated that the multiclass SVMs trained on extracted features achieved high accuracy rates.     

Analysis of electrocardiographic changes in partial epileptic patients by combining eigenvector methods and support vector machines

Abstract: In the present study, the diagnostic accuracy of support vector machines (SVMs) on electrocardiogram (ECG) signals is evaluated. Two types of ECG beats (normal and partial epilepsy) were obtained from the Physiobank database. Decision making was performed in two stages: feature extraction by eigenvector methods and classification using the SVM trained on the extracted features. The present research demonstrates that the power levels of the power spectral densities obtained by eigenvector methods are features which represent the ECG signals well and SVMs trained on these features achieve high classification accuracies.     

EEG signal classification using wavelet feature extraction and a mixture of expert model

Mixture of experts (ME) is modular neural network architecture for supervised learning. A double-loop Expectation-Maximization (EM) algorithm has been introduced to the ME network structure for detection of epileptic seizure. The detection of epileptiform discharges in the EEG is an important component in the diagnosis of epilepsy. EEG signals were decomposed into the frequency sub-bands using discrete wavelet transform (DWT). Then these sub-band frequencies were used as an input to a ME network with two discrete outputs: normal and epileptic. In order to improve accuracy, the outputs of expert networks were combined according to a set of local weights called the “gating function”. The invariant transformations of the ME probability density functions include the permutations of the expert labels and the translations of the parameters in the gating functions. The performance of the proposed model was evaluated in terms of classification accuracies and the results confirmed that the proposed ME network structure has some potential in detecting epileptic seizures. The ME network structure achieved accuracy rates which were higher than that of the stand-alone neural network model.     

Implementing wavelet transform/mixture of experts network for analysis of electrocardiogram beats

Abstract: Mixture of experts (ME) is a modular neural network architecture for supervised learning. This paper illustrates the use of the ME network structure to guide model selection for classification of electrocardiogram (ECG) beats. The expectation maximization algorithm is used for training the ME so that the learning process is decoupled in a manner that fits well with the modular structure. The ECG signals were decomposed into time–frequency representations using discrete wavelet transforms and statistical features were calculated to depict their distribution. The ME network structure was implemented for ECG beats classification using the statistical features as inputs. To improve classification accuracy, the outputs of expert networks were combined by a gating network simultaneously trained in order to stochastically select the expert that is performing the best at solving the problem. Five types of ECG beats (normal beat, congestive heart failure beat, ventricular tachyarrhythmia beat, atrial fibrillation beat, partial epilepsy beat) obtained from the Physiobank database were classified with an accuracy of 96.89% by the ME network structure. The ME network structure achieved accuracy rates which were higher than those of the stand-alone neural network models.     

Signal‐to‐noise ratios for measuring saliency of features extracted by eigenvector methods from ophthalmic arterial Doppler signals

Abstract: Features are used to represent patterns with minimal loss of important information. The feature vector, which is composed of the set of all features used to describe a pattern, is a reduced‐dimensional representation of that pattern. Medical diagnostic accuracies can be improved when the pattern is simplified through representation by important features. By identifying a set of salient features, the noise in a classification model can be reduced, resulting in more accurate classification. In this study, a signal‐to‐noise ratio saliency measure was employed to determine the saliency of input features of recurrent neural networks (RNNs) used in classification of ophthalmic arterial Doppler signals. Eigenvector methods were used to extract features representing the ophthalmic arterial Doppler signals. The RNNs used in the ophthalmic arterial Doppler signal classification were trained for the signal‐to‐noise ratio screening method. The application results of the signal‐to‐noise ratio screening method to the ophthalmic arterial Doppler signals demonstrated that classification accuracies of RNNs with salient input features are higher than those of RNNs with salient and non‐salient input features.     

Feature saliency using signal-to-noise ratios in automated diagnostic systems developed for Doppler ultrasound signals

Artificial neural networks (ANNs) have been used in a great number of medical diagnostic decision support system applications and within feedforward ANNs framework there are a number of established measures such as saliency measures for identifying important input features. By identifying a set of salient features, the noise in a classification model can be reduced, resulting in more accurate classification. In this study, a signal-to-noise ratio (SNR) saliency measure was employed to determine saliency of input features of multilayer perceptron neural networks (MLPNNs) used in classification of Doppler signals. The SNR saliency measure determines the saliency of a feature by comparing it to that of an injected noise feature and the SNR screening method utilizes the SNR saliency measure to select a parsimonious set of salient features. Ophthalmic and internal carotid arterial Doppler signals were decomposed into time–frequency representations using discrete wavelet transform. Input feature vectors were extracted using statistics over the set of the wavelet coefficients. The MLPNNs used in classification of the ophthalmic and internal carotid arterial Doppler signals were trained for the SNR screening method. The application results of the SNR screening method to the ophthalmic and internal carotid arterial Doppler signals demonstrated that classification accuracies of the MLPNNs with salient input features are higher than that of the MLPNNs with salient and non-salient input features.     

Recurrent neural networks employing Lyapunov exponents for analysis of doppler ultrasound signals

The implementation of recurrent neural networks (RNNs) with the Lyapunov exponents for Doppler ultrasound signals classification is presented. This study is based on the consideration that Doppler ultrasound signals are chaotic signals. This consideration was tested successfully using the nonlinear dynamics tools, like the computation of Lyapunov exponents. Decision making was performed in two stages: computation of Lyapunov exponents as representative features of the Doppler ultrasound signals and classification using the RNNs trained on the extracted features. The present research demonstrated that the Lyapunov exponents are the features which well represent the Doppler ultrasound signals and the RNNs trained on these features achieved high classification accuracies.     

Probabilistic neural networks combined with wavelet coefficients for analysis of electroencephalogram signals

Abstract: In this paper, the probabilistic neural network is presented for classification of electroencephalogram (EEG) signals. Decision making is performed in two stages: feature extraction by wavelet transform and classification using the classifiers trained on the extracted features. The purpose is to determine an optimum classification scheme for this problem and also to infer clues about the extracted features. The present research demonstrates that the wavelet coefficients obtained by the wavelet transform are features which represent the EEG signals well. The conclusions indicate that the probabilistic neural network trained on the wavelet coefficients achieves high classification accuracies (the total classification accuracy is 97.63%).     

基于多特征组合的细粒度图像分类方法

针对单一特征表示的局限性会导致细粒度图像分类准确度不高的问题,提出了一种基于卷积神经网络（CNN）和尺度不变特征转换（SIFT）的多特征组合表示方法,综合考虑对目标整体、关键部位和关键点的特征提取。首先,分别以细粒度图像库中的目标整体和头部区域训练CNN得到两个网络模型,用来提取目标的整体和头部CNN特征;然后,对图像库中所有目标区域提取SIFT关键点并通过K均值（K-means）聚类生成码本,再将每个目标区域的SIFT描述子通过局部特征聚合描述符（VLAD）参照码本编码为特征向量;最后,组合多种特征作为最终的特征表示,采用支持向量机（SVM）对细粒度图像进行分类。使用该方法在CUB-200-2011数据库上进行实验,并与单一的特征表示方法进行了比较。实验结果表明,该方法与基于单一CNN特征的细粒度图像分类相比提升了13.31%的准确度,证明了多特征组合对细粒度图像分类的积极作用。     

Measuring saliency of features representing EEG signals using signal-to-noise ratios

Various methodologies of automated diagnosis have been adopted, however the entire process can generally be subdivided into a number of disjoint processing modules: pre-processing, feature extraction/selection, and classification. Features are used to represent patterns with minimal loss of important information. The feature vector, which is comprised of the set of all features used to describe a pattern, is a reduced-dimensional representation of that pattern. Medical diagnostic accuracies can be improved when the pattern is simplified through representation by important features. By identifying a set of salient features, the noise in a classification model can be reduced, resulting in more accurate classification. In this study, a signal-to-noise ratio (SNR) saliency measure was employed to determine saliency of input features of probabilistic neural networks (PNNs) used in classification of electroencephalogram (EEG) signals. In order to extract features representing the EEG signals, eigenvector methods were used. The PNNs used in the EEG signals classification were trained for the SNR screening method. The application results of the SNR screening method to the EEG signals demonstrated that classification accuracies of the PNNs with salient input features are higher than that of the PNNs with salient and non-salient input features.     

Combined neural network model to compute wavelet coefficients

Abstract: In recent years a novel model based on artificial neural networks technology has been introduced in the signal processing community for modelling the signals under study. The wavelet coefficients characterize the behaviour of the signal and computation of the wavelet coefficients is particularly important for recognition and diagnostic purposes. Therefore, we dealt with wavelet decomposition of time-varying biomedical signals. In the present study, we propose a new approach that takes advantage of combined neural network (CNN) models to compute the wavelet coefficients. The computation was provided and expressed by applying the CNNs to ophthalmic arterial and internal carotid arterial Doppler signals. The results were consistent with theoretical analysis and showed good promise for discrete wavelet transform of the time-varying biomedical signals. Since the proposed CNNs have high performance and require no complicated mathematical functions of the discrete wavelet transform, they were found to be effective for the computation of wavelet coefficients.     

Combining features of negative correlation learning with mixture of experts in proposed ensemble methods

Both theoretical and experimental studies have shown that combining accurate neural networks (NNs) in the ensemble with negative error correlation greatly improves their generalization abilities. Negative correlation learning (NCL) and mixture of experts (ME), two popular combining methods, each employ different special error functions for the simultaneous training of NNs to produce negatively correlated NNs. In this paper, we review the properties of the NCL and ME methods, discussing their advantages and disadvantages. Characterization of both methods showed that they have different but complementary features, so if a hybrid system can be designed to include features of both NCL and ME, it may be better than each of its basis approaches. In this study, two approaches are proposed to combine the features of both methods in order to solve the weaknesses of one method with the strength of the other method, i.e., gated-NCL (G-NCL) and mixture of negatively correlated experts (MNCE). In the first approach, G-NCL, a dynamic combiner of ME is used to combine the outputs of base experts in the NCL method. The suggested combiner method provides an efficient tool to evaluate and combine the NCL experts by the weights estimated dynamically from the inputs based on the different competences of each expert regarding different parts of the problem. In the second approach, MNCE, the capability of a control parameter for NCL is incorporated in the error function of ME, which enables the training algorithm of ME to efficiently adjust the measure of negative correlation between the experts. This control parameter can be regarded as a regularization term added to the error function of ME to establish better balance in bias–variance–covariance trade-offs and thus improves the generalization ability. The two proposed hybrid ensemble methods, G-NCL and MNCE, are compared with their constituent methods, ME and NCL, in solving several benchmark problems. The experimental results show that our proposed methods preserve the advantages and alleviate the disadvantages of their basis approaches, offering significantly improved performance over the original methods.     

基于CNN和BiLSTM网络特征融合的文本情感分析

卷积神经网络（CNN）和循环神经网络（RNN）在自然语言处理上得到广泛应用,但由于自然语言在结构上存在着前后依赖关系,仅依靠卷积神经网络实现文本分类将忽略词的上下文含义,且传统的循环神经网络存在梯度消失或梯度爆炸问题,限制了文本分类的准确率。为此,提出一种卷积神经网络和双向长短时记忆（BiLSTM）特征融合的模型,利用卷积神经网络提取文本向量的局部特征,利用BiLSTM提取与文本上下文相关的全局特征,将两种互补模型提取的特征进行融合,解决了单卷积神经网络模型忽略词在上下文语义和语法信息的问题,也有效避免了传统循环神经网络梯度消失或梯度弥散问题。在两种数据集上进行对比实验,实验结果表明,所提特征融合模型有效提升了文本分类的准确率。     

Combining recurrent neural networks with eigenvector methods for classification of ECG beats

The purpose of this study is to evaluate the accuracy of the recurrent neural networks (RNNs) trained with Levenberg–Marquardt algorithm on the electrocardiogram (ECG) beats. Four types of ECG beats (normal beat, congestive heart failure beat, ventricular tachyarrhythmia beat, atrial fibrillation beat) obtained from the Physiobank database were analyzed. Decision making was performed in two stages: feature extraction by eigenvector methods and classification using the RNN trained on the extracted features. The RNNs were implemented for classification of the ECG beats using the statistical features as inputs. The ability of designed and trained Elman RNNs, combined with eigenvector methods, were explored to classify the ECG beats. The classification results demonstrated that the combined eigenvector methods/RNN approach can be useful in analyzing the ECG beats.     

基于深度学习的计算机显示器电磁信息泄漏识别

本文以计算机显示设备泄漏电磁信号为研究对象,对于人工提取特征识别电磁泄漏信号存在的主观性强、特征冗余的问题,区别于传统基于经验的人工特征提取模式,利用人工智能深度学习方法,使用处理图像的深度学习技术应用于电磁信息泄漏特征识别,提出了一种基于卷积神经网络的识别方法.该方法首先提取电磁泄漏信号的时频谱信息作为卷积神经网络模型的输入,然后利用模型的自学习能力提取深层特征,实现对不同分辨率来源电磁泄漏信号的识别,识别准确率达到98％,单信号检测时间仅需40 ms,验证了卷积神经网络应用于电磁泄漏信号识别的有效性,为电磁泄漏预警与防护提供了重要依据,为电磁泄漏视频信号还原复现提供有力支撑.     

Medical diagnosis of rheumatoid arthritis disease from right and left hand Ulnar artery Doppler signals using adaptive network based fuzzy inference system (ANFIS) and MUSIC method

Rheumatoid arthritis (RA) is a multi-systemic autoimmune disease that leads to substantial morbidity and mortality. In this paper, as spectral analysis methods of Multiple Signal Classification (MUSIC) method is used in order to extract the significant features from the right and left hand Ulnar artery Doppler signals for the diagnosis of RA disease. The MUSIC method has been used as subspace method. To extract features from Doppler signals obtained from the right and left hand Ulnar arterial the MUSIC method model degrees of 5, 10, 15, 20, and 25 were used. Then, an adaptive network based fuzzy inference system (ANFIS) was applied to features extracted from the right and left hand Ulnar artery Doppler signals for classifying RA disease. The methods are not new, but the study has a novelty in that the application area of these methods is new. In the hybrid model, the combination of MUSIC and ANFIS yielded classification accuracies of 95% (for a model degree of 20) using the right hand Ulnar artery and classification accuracies of 91.25% (for a model degree of 10) using left hand Ulnar artery Doppler signals in the diagnosis of RA disease. The proposed approach has potential to help with the early diagnosis of RA disease for the specialists who study this subject.     

稀疏贝叶斯混合专家模型及其在光谱数据标定中的应用

在光谱数据的多元校正中, 光谱数据通常是在多种不同的环境条件下收集的. 为了建模来源于不同环境中的高维光谱数据, 本文提出了一种新的稀疏贝叶斯混合专家模型, 并将其用来选择多元校正模型的稀疏特征. 混合专家模型能够把训练数据划分到不同的子类, 之后使用不同的预测模型来分别对划分后的数据进行预测, 因此这种方法适合于建模来自于多种环境下的光谱数据. 本文提出的稀疏的混合专家模型利用稀疏贝叶斯的方法来进行特征选择, 不依赖于事先指定的参数; 同时利用probit模型作为门函数以得到解析的后验分布, 避免了在门函数分类模型中进行特征提取时需要的近似. 本文提出的模型与其他几种常用的回归模型在人工数据集和几个公开的光谱数据集上进行了比较, 比较结果显示本文提出的模型对多个来源的光谱数据进行浓度预测时精度比传统的回归方法有一定的提高.