基于堆叠降噪自编码器的神经–符号模型及在晶圆表面缺陷识别

深度神经网络是具有复杂结构和多个非线性处理单元的模型, 通过模块化的方式分层从数据提取代表性特征, 已经在晶圆缺陷识别领域得到了较为广泛的应用. 但是, 深度神经网络在应用过程中本身存在“黑箱”和过度依赖数据的问题, 显著地影响深度神经网络在晶圆缺陷识别的工业可应用性. 提出一种基于堆叠降噪自编码器的神经–符号模型. 首先, 根据堆叠降噪自编码器的网络特点采用了一套符号规则系统, 规则形式和组成结构使其可与深度神经网络有效融合. 其次, 根据 网络和符号规则之间的关联性提出完整的知识抽取与插入算法, 实现了深度网络和规则之间的知识转换. 在实际工业晶圆表面图像数据集WM-811K上的试验结果表明, 基于堆叠降噪自编码器的神经–符号模型不仅取得了较好的缺陷探测与识别性能, 而且可有效提取规则并通过规则有效描述深度神经网络内部计算逻辑, 综合性能优于目前经典的深度神经网络.     

深度区域网络方法的细粒度图像分类

目的 在细粒度视觉识别中,难点是对处于相同层级的大类,区分其具有微小差异的子类,为实现准确的分类精度,通常要求具有专业知识,所以细粒度图像分类为计算机视觉的研究提出更高的要求。为了方便普通人在不具备专业知识和专业技能的情况下能够区分物种细粒度类别,进而提出一种基于深度区域网络的卷积神经网络结构。方法 该结构基于深度区域网络,首先,进行深度特征提取任务,使用VGG16层网络和残差101层网络两种结构作为特征提取网络,用于提取深层共享特征,产生特征映射。其次,使用区域建议网络结构,在特征映射上进行卷积,产生目标区域;同时使用兴趣区域（RoI）池化层对特征映射进行最大值池化,实现网络共享。之后将池化后的目标区域输入到区域卷积网络中进行细粒度类别预测和目标边界回归,最终输出网络预测类别及回归边框点坐标。同时还进行了局部遮挡实验,检测局部遮挡部位对于分类正确性的影响,分析局部信息对于鸟类分类的影响情况。结果 该模型针对CUB_200_2011鸟类数据库进行实验,该数据库包含200种细粒度鸟类类别,11 788幅鸟类图片。经过训练及测试,实现VGG16+R-CNN （RPN）和Res101+R-CNN （RPN）两种结构验证正确率分别为90.88%和91.72%,两种结构Top-5验证正确率都超过98%。本文模拟现实环境遮挡情况进行鸟类局部特征遮挡实验,检测分类效果。结论 基于深度区域网络的卷积神经网络模型,提高了细粒度鸟类图像的分类性能,在细粒度鸟类图像的分类上,具有分类精度高、泛化能力好和鲁棒性强的优势,实验发现头部信息对于细粒度鸟类分类识别非常重要。     

Principal feature classification

The concept, structures, and algorithms of principal feature classification (PFC) are presented in this paper. PFC is intended to solve complex classification problems with large data sets. A PFC network is designed by sequentially finding principal features and removing training data which has already been correctly classified. PFC combines advantages of statistical pattern recognition, decision trees, and artificial neural networks (ANNs) and provides fast learning with good performance and a simple network structure. For the real-world applications of this paper, PFC provides better performance than conventional statistical pattern recognition, avoids the long training times of backpropagation and other gradient-descent algorithms for ANNs, and provides a low-complexity structure for realization.     

Fuzzy feature selection

In fuzzy classifier systems the classification is obtained by a number of fuzzy If–Then rules including linguistic terms such as Low and High that fuzzify each feature. This paper presents a method by which a reduced linguistic (fuzzy) set of a labeled multi-dimensional data set can be identified automatically. After the projection of the original data set onto a fuzzy space, the optimal subset of fuzzy features is determined using conventional search techniques. The applicability of this method has been demonstrated by reducing the number of features used for the classification of four real-world data sets. This method can also be used to generate an initial rule set for a fuzzy neural network.     

基于双池化特征加权结构CNN的图像分类

传统的池化方式会造成特征信息丢失,导致卷积神经网络中提取的特征信息不足。为了提高卷积神经网络在图像分类过程中的准确率,优化其学习性能,本文在传统池化方式的基础上提出一种双池化特征加权结构的池化算法,利用最大池化和平均池化2种方式保留更多的有价值的特征信息,并通过遗传算法对模型进行优化。通过训练不同池化方式的卷积神经网络,研究卷积神经网络在不同数据集上的分类准确率和收敛速度。实验在遥感图像数据集NWPU-RESISC45和彩色图像数据集Cifar-10上对采用几种池化方式的卷积神经网络分类结果进行对比验证,结果分析表明：双池化特征加权结构使得卷积神经网络的分类准确率有很大程度的提高,同时模型的收敛速度得到进一步提高。     

一个基于模糊神经网络的模式分类系统

目前,基于神经网络的分类系统在许多领域得到了越来越广泛的应用。但是,该系统大多采用的是离线自适应机制,即神经网络需学习新的分类知识时,要重新训练神经网络,从而大大增加神经网络的训练时间;对于重叠分类,一般是构成一个贝叶斯分类器。然而,贝叶斯分类器的构成需要关于分类数据的概率密度函数的先验知识,而这些知识常常在模式分类前是难以获得的。为了解决这些问题,文中根据模糊集合理论,提出了一种基于模糊神经网络     

Possibility-based fuzzy neural networks and their application toimage processing

This paper describes the foundations for a class of fuzzy neural networks. Such a network is a composite or two-stage network consisting of a fuzzy network stage and a neural network stage. It exhibits the ability to classify complex feature set vectors with a configuration that is simpler than that needed by a standard neural network, Unlike a standard neural network, this network is able to accept as input a vector of scalar values, or a vector (set) of possibility functions. The first stage of the network is fuzzy based. It has two parts: a parameter computing network (PCN), followed by a converting layer. In the PCN the weights of the nodes are possibility functions, and hence, the output of this network is a fuzzy set. The second part of this stage, which is a single layer network, then converts this fuzzy set into a scalar vector for input to the second stage. The second stage of the network is a standard backpropagation based neural network. In addition to establishing the theoretical foundations for such a network, this paper presents sample applications of the network for classification problems in satellite image processing and seismic lithology pattern recognition.     

基于优选小波基和模糊SOM网络的模拟电路故障诊断

为了解决模拟电路故障诊断中的特征提取困难并实现对模拟电路故障模式准确的分类,提出一种优选小波基、模糊理论和自组织特征映射网络(SOM,self-organizing feature map)相结合的模拟电路故障诊断方法.该方法首先对模拟电路故障响应信号进行小波分解、提取能量值、均值和方差组成输入特征向量,同时采用余弦分离度评价小波变换在不同小波基函数下获取故障特征的有效性,据此选择余弦分离度最小的小波基分解的特征向量输入到自组织特征映射网络进行故障分类.仿真实验表明,利用余弦分离度选择的最优小波基能有效提高模拟电路故障特征提取,模糊神经网络能对故障模式进行精确分类.     

A new genetic feature selection with neural network ensemble

A neural network ensemble is a learning paradigm in which a finite collection of neural networks is trained for the same task. Ensembles generally show better classification and generalization performance than a single neural network does. In this paper, a new feature selection method for a neural network ensemble is proposed for pattern classification. The proposed method selects an adequate feature subset for each constituent neural network of the ensemble using a genetic algorithm. Unlike the conventional feature selection method, each neural network is only allowed to have some (not all) of the considered features. The proposed method can therefore be applied to huge-scale feature classification problems. Experiments are performed with four databases to illustrate the performance of the proposed method.     

Fuzzy min-max neural networks. I. Classification.

A supervised learning neural network classifier that utilizes fuzzy sets as pattern classes is described. Each fuzzy set is an aggregate (union) of fuzzy set hyperboxes. A fuzzy set hyperbox is an n-dimensional box defined by a min point and a max point with a corresponding membership function. The min-max points are determined using the fuzzy min-max learning algorithm, an expansion-contraction process that can learn nonlinear class boundaries in a single pass through the data and provides the ability to incorporate new and refine existing classes without retraining. The use of a fuzzy set approach to pattern classification inherently provides a degree of membership information that is extremely useful in higher-level decision making. The relationship between fuzzy sets and pattern classification is described. The fuzzy min-max classifier neural network implementation is explained, the learning and recall algorithms are outlined, and several examples of operation demonstrate the strong qualities of this new neural network classifier.     

A HYBRID INTELLIGENT MODEL BASED ON EVOLUTIONARY FUZZY CLUSTERING AND SYNDICATE NEURAL NETWORKS

In this article, a new hybrid intelligent model comprising a cluster allocation and adaptation component is developed for solving classification and pattern recognition problems. Its computation ability has been verified through various benchmark problems and biometric applications. The proposed model consists of two components: cluster distribution and adaptation. In the first module, mean patterns are distributed into the number of clusters based on the evolutionary fuzzy clustering, which is the basis for network structure selection in next module. In the second module, training and subsequent generalization is performed by the syndicate neural networks (SNN). The number of SNNs required in the second module will be same as the number of clusters. Whereas each network contains as many output neurons as the maximum number of members assigned to each cluster. The proposed novel fusion of evolutionary fuzzy clustering with a neural network yields superior performance in classification and pattern recognition problems. Performance evaluation has been carried out over a wide spectrum of benchmark problems and real-life biometric recognition problems with noise and occlusion. Experimental results demonstrate the efficacy of the methodology over existing ones.     

Layer-wise domain correction for unsupervised domain adaptation

Deep neural networks have been successfully applied to numerous machine learning tasks because of their impressive feature abstraction capabilities. However, conventional deep networks assume that the training and test data are sampled from the same distribution, and this assumption is often violated in real-world scenarios. To address the domain shift or data bias problems, we introduce layer-wise domain correction (LDC), a new unsupervised domain adaptation algorithm which adapts an existing deep network through additive correction layers spaced throughout the network. Through the additive layers, the representations of source and target domains can be perfectly aligned. The corrections that are trained via maximum mean discrepancy, adapt to the target domain while increasing the representational capacity of the network. LDC requires no target labels, achieves state-of-the-art performance across several adaptation benchmarks, and requires significantly less training time than existing adaptation methods.     

Opening the black box of neural networks for remote sensing image classification

Neural networks, which make no assumption about data distribution, have achieved improved image classification results compared to traditional methods. Unfortunately, a neural network is generally perceived as being a ‘black box’. It is extremely difficult to document how specific classification decisions are reached. Fuzzy systems, on the other hand, have the capability to represent classification decisions explicitly in the form of fuzzy ‘if-then’ rules. However, the construction of a knowledge base, especially the fine-tuning of the fuzzy set parameters of the fuzzy rules in a fuzzy expert system, is a tedious and subjective process. This research has developed a new, improved neuro-fuzzy image classification system based on the synergism between neural networks and fuzzy expert systems. It incorporates the best of both technologies and compensates for the shortcomings of each. The learning algorithms of neural networks developed here are used to automate the derivation of fuzzy set parameters for the fuzzy ‘if-then’ rules in a fuzzy expert system. The rules obtained, in symbolic form, facilitate the understanding of the neural network based image classification system. In addition, the image classification accuracy obtained from the improved neuro-fuzzy system was significantly superior to those of the back-propagation based neural network and the maximum likelihood approaches.     

Underwater target classification in changing environments using an adaptive feature mapping

A new adaptive underwater target classification system to cope with environmental changes in acoustic backscattered data from targets and nontargets is introduced. The core of the system is the adaptive feature mapping that minimizes the classification error rate of the classifier. The goal is to map the feature vector in such a way that the mapped version remains invariant to the environmental changes. A K-nearest neighbor (K-NN) system is used as a memory to provide the closest matches of an unknown pattern in the feature space. The classification decision is done by a backpropagation neural network (BPNN). Two different cost functions for adaptation are defined. These two cost functions are then combined together to improve the classification performance. The test results on a 40-kHz linear FM acoustic backscattered data set collected from six different objects are presented. These results demonstrate the effectiveness of the adaptive system versus nonadaptive system when the signal-to-reverberation ratio (SRR) in the environment is varying.     

基于多时空图卷积网络的交通流预测

交通流预测在交通管理和城市规划的应用中具有重要意义,然而现有的预测方法无法充分挖掘其潜在的复杂时空相关性,为进一步挖掘路网道路网络数据的时空特性以提高预测精度,提出一种多时空图卷积网络(multi-spatial-temporal graph convolutional network,MST-GCN)模型。首先,利用切比雪夫图卷积(ChebNet)结合门控循环单元(GRU)构建时空组件以深度挖掘节点的时空相关性;其次,分别提取周相关、日相关、邻近时间的序列数据,输入三个时空组件以深度挖掘不同时间窗口间的时间相关性;最后,将时空组件与编码器—解码器网络结构(encoder-decoder)融合组建MST-GCN模型。利用加利福尼亚州交通局(Caltrans)性能评估系统中高速公路数据集PEMS04和PEMS08进行实验,结果表明新模型的性能明显优于门控循环单元模型和最近提出的扩散卷积循环神经网络(DCRNN)、时间图卷积网络(T-GCN)、基于注意力机制的时空图卷积神经网络(ASTGCN)和时空同步图卷积网络(STSGCN)模型。     

Quantum neural networks (QNNs): inherently fuzzy feedforward neuralnetworks

This paper introduces quantum neural networks (QNNs), a class of feedforward neural networks (FFNNs) inherently capable of estimating the structure of a feature space in the form of fuzzy sets. The hidden units of these networks develop quantized representations of the sample information provided by the training data set in various graded levels of certainty. Unlike other approaches attempting to merge fuzzy logic and neural networks, QNNs can be used in pattern classification problems without any restricting assumptions such as the availability of a priori knowledge or desired membership profile, convexity of classes, a limited number of classes, etc. Experimental results presented here show that QNNs are capable of recognizing structures in data, a property that conventional FFNNs with sigmoidal hidden units lack.     

Comparing neural networks: a benchmark on growing neural gas,growing cell structures, and fuzzy ARTMAP

Compares the performance of some incremental neural networks with the well-known multilayer perceptron (MLP) on real-world data. The incremental networks are fuzzy ARTMAP (FAM), growing neural gas (GNG) and growing cell structures (GCS). The real-world datasets consist of four different datasets posing different challenges to the networks in terms of complexity of decision boundaries, overlapping between classes, and size of the datasets. The performance of the networks on the datasets is reported with respect to measure classification error, number of training epochs, and sensitivity toward variation of parameters. Statistical evaluations are applied to examine the significance of the results. The overall performance ranks in the following descending order: GNG, GCS, MLP, FAM.     

Multi-class pattern classification using neural networks

Multi-class pattern classification has many applications including text document classification, speech recognition, object recognition, etc. Multi-class pattern classification using neural networks is not a trivial extension from two-class neural networks. This paper presents a comprehensive and competitive study in multi-class neural learning with focuses on issues including neural network architecture, encoding schemes, training methodology and training time complexity. Our study includes multi-class pattern classification using either a system of multiple neural networks or a single neural network, and modeling pattern classes using one-against-all, one-against-one, one-against-higher-order, and P-against-Q. We also discuss implementations of these approaches and analyze training time complexity associated with each approach. We evaluate six different neural network system architectures for multi-class pattern classification along the dimensions of imbalanced data, large number of pattern classes, large vs. small training data through experiments conducted on well-known benchmark data.     

Spatio-temporal feature maps using gated neuronal architecture

In this paper, Kohonen's self-organizing feature map is modified by a novel technique of allowing the neurons in the feature map to compete in a selective manner. The selective competition is achieved by grating the N-dimensional feature space using a spatial frequency and setting a criterion for the neurons to compete based on the region in which the input pattern resides. The spatial grating and selective competition are achieved by introducing a gated neuronal architecture in the feature map. As the selection criterion changes with time, it generates a time sequence of winning node indexes providing more input information and potentially allowing higher classification performance. These time sequences are then used to predict the class label of the input pattern more accurately. Three possible class label prediction algorithms are formulated based on evidential reasoning method and Bayes conditional probability theorem. These are tested on real world 8-class texture and a synthetic 12-class 3D object recognition problems. The classification performance is then compared with the results obtained by using a standard statistical linear discriminant analysis.     

DenseNet-GRU:直肠癌CT影像分类的深度神经网络模型

DenseNet是一种广泛用于影像分类的卷积神经网络,但它不具备记忆功能,无法反映卷积操作后不同特征映射之间的关联关系。若将其直接应用于判断直肠癌是否发生淋巴结转移,则无法比较直肠癌CT影像特征在深度神经网络映射过程中的变化。基于此,提出了一种新颖的深度神经网络模型DenseNet-GRU(gated recurrent unit),其核心是利用GRU获取DenseNet提取的不同影像特征之间的关联关系,进而获得不同图像之间相同像素区域的特征变化情况,最终判断直肠癌患者的淋巴结是否存在转移。以包含107个患者DCM格式的腹部横断位动脉期和门脉期两种增强CT影像为实验数据集,采用数据增强和阈值分割方法对数据进行预处理,DenseNet-GRU模型在F-score上的分类精度达到了65%以上,对临床辅助诊断具有重要的现实意义。