神经网络模型的透明化及输入变量约简

由于神经网络很容易实现从输入空间到输出空间的非线性映射,因此,神经网络应用者往往未考虑输入变量和输出变量之间的相关性,直接用神经网络来实现输入变量与输出变量之间的黑箱建模,致使模型中常存在冗余变量,并造成模型可靠性和鲁棒性差。提出一种透明化神经网络黑箱特性的方法,并用它剔除模型中的冗余变量。该方法首先利用神经网络释义图可视化网络;再利用连接权法计算神经网络输入变量的相对贡献率,判断其对输出变量的重要性;最后利用改进的随机化测验对连接权和输入变量贡献率进行显著性检验,修剪模型,并以综合贡献度和相对贡献率均不显著的输入变量的交集为依据,剔除冗余变量,实现NN模型透明化及变量选择。实验结果表明,该方法增加了模型的透明度,选择出了最佳输入变量,剔除了冗余输入变量,提高了模型的可靠性和鲁棒性。因此,该研究为神经网络模型的透明化及变量约简提供了一种新的方法。     

基于免疫算法与支持向量机的异常检测方法

在异常检测中, 应用支持向量机算法能使检测系统在小样本的条件下具有良好的泛化能力。 但支持向量机的参数取值决定了其学习性能和泛化能力，且大量无关或冗余的特征会降低分类的性能。基于此，提出了一种基于免疫算法的支持向量机参数和特征选择联合优化的方法。免疫算法是一种新的有效随机全局优化技术，它具有不易陷入局部最优、解的精度高、收敛速度快等优点。 仿真结果表明算法在提高异常检测的检测正确率的同时相应的测试时间也在缩短。     

针对Lasso问题的多维权重求解算法

最小绝对收缩和选择算子（Lasso）在数据维度约减、异常检测方面有着较强的计算优势。针对Lasso用于异常检测中检测精度不高的问题,提出了一种基于多维度权重的最小角回归（LARS）算法解决Lasso问题。首先考虑每个回归变量在回归模型中所占权重不同,即此属性变量在整体评价中的相对重要程度不同,故在LARS算法计算角分线时,将各回归变量与剩余变量的联合相关度纳入考虑,用来区分不同属性变量对检测结果的影响;然后在LARS算法中加入主成分分析（PCA）、独立权数法、基于Intercriteria相关性的指标的重要度评价（CRITIC）法这三种权重估计方法,并进一步对LARS求解的前进方向和前进变量选择进行优化。最后使用Pima Indians Diabetes数据集验证算法的优良性。实验结果表明,在更小阈值的约束条件下,加入多维权重后的LARS算法对Lasso问题的解具有更高的准确度,能更好地用于异常检测。     

基于互信息的分步式输入变量选择多元序列预测研究

针对多元序列分析中存在的输入变量选择问题,提出一种基于k-!近邻互信息估计的分步式变量选择算法. 该算法通过两步过程分别实现相关变量的选择与弱相关变量的剔除. 同时将分步变量选择算法应用于径向基函数(Radial basis function, RBF) 神经网络结构的优化中.在K均值聚类的基础上,通过分析隐含层神经元的输出权值与神经网络输出的相关性, 对隐含层节点进行选择,改进网络的结构与性能. Friedman数据的仿真实验验证了分步变量选择算法的有效性; Gas furnace多元时间序列以及Boston housing数据的仿真结果表明, 优化后的RBF网络能够在保证模型精度的基础上有效控制网络规模.     

Importance analysis for models with correlated input variables by the state dependent parameters method

For clearly exploring the origin of the variance of the output response in case the correlated input variables are involved, a novel method on the state dependent parameters (SDP) approach is proposed to decompose the contribution by correlated input variables to the variance of output response into two parts: the uncorrelated contribution due to the unique variations of a variable and the correlated one due to the variations of a variable correlated with other variables. The correlated contribution is composed by the components of the individual input variable correlated with each of the other input variables. An effective and simple SDP method in concept is further proposed to decompose the correlated contribution into the components, on which a second order importance matrix can be solved for explicitly exposing the contribution components of the correlated input variable to the variance of the output response. Compared with the existing regression-based method for decomposing the contribution by correlated input variables to the variance of the output response, the proposed method is not only applicable for linear response functions, but is also suitable for nonlinear response functions. It has advantages both in efficiency and accuracy, which are demonstrated by several numerical and engineering examples.     

基于遗传算法的入侵检测特征选择*

针对入侵检测日志数据存在大量不相关特征和冗余特征,导致入侵检测数据集维数较高,检测算法实时性较低的问题,提出一种基于遗传算法的入侵检测特征选择算法。首先删除入侵检测数据集中的不相关特征及冗余特征,构建有效特征集L,并通过偏F检验对特征进一步选择,构成待优化特征集L’;然后采用遗传算法对L’进行优化选择,选出最能反映系统状态的特征集L″。仿真实验结果证明,该算法在保证特征分类精度和确保入侵检测漏检率、误检率尽量小的前提下明显提高了入侵检测的效率。     

基于特征选择的网络入侵检测方法

针对现有入侵检测算法中存在着冗余或噪音特征导致的检测模型精度下降与训练时间过长的问题进行了研究,将特征选择算法引入到入侵检测领域,提出了一种基于特征选择的入侵检测方法.利用不同的离散化与特征选择算法生成具有差异的多个最优特征子集,并对每个特征子集进行归一化处理,用分类算法对提取后的特征进行学习建模.通过实验将该方法与基于传统算法（决策树、朴素贝叶斯、支持向量机）的入侵检测方法作比较,实验结果表明,该方法有效地提高了检测攻击的准确率,并且降低了模型的训练时间.     

一类不确定非线性MIMO系统的神经网络输出反馈跟踪控制

针对一类具有外部干扰的不确定仿射非线性MIMO系统提出了一种神经网络输出反馈跟踪控制方法. 在仅输出可测的情况下, 控制律和神经网络权值更新律中仅用到输出误差, 无需设计状态观测器或加入低通滤波器使得估计误差动态满足严格正实条件. 为抑制外部干扰和子系统间的交叉耦合及神经网络逼近误差, 在控制律中加入鲁棒控制项. 基于Lyapunov稳定性定理证明了系统的稳定性及信号的有界性. 仿真例子证实了所提方法的可行性.     

快速判定几何约束奇异性的切面扰动法

针对冗余奇异和分支奇异的判定问题,提出一种新的切面扰动的判定方法.该方法将奇异的雅可比矩阵分为独立构型空间和奇异空间,变量沿独立构型空间的切面扰动,计算更新的雅克比矩阵的秩,依据秩亏的变化可以快速、稳定地判定约束奇异性.该算法克服了残量扰动法的数值迭代、计算量大和不稳定的缺点,并且在参数化特征造型系统InteSolid中得到验证.     

Point estimate-based importance analysis for structural models with correlated variables

Importance analysis is conducted to find the contributions of the inputs to the output uncertainty. In this work, a point estimate-based importance analysis algorithm is established for models involving correlated input variables, and the variance contribution by an individual correlated input variable is decomposed into correlated contribution and uncorrelated contribution. In the established algorithm, the correlated variables are orthogonalised to generate corresponding independent variables, and the performance function is reconstructed in the independence space. Then, the point estimate is employed to compute the variance-based importance measures in the independence space, by which the variance contribution of the original correlated variables, including the correlated part and uncorrelated part, can be obtained. Different point estimate methods can be employed in the proposed algorithm; thus, the algorithm is adaptable and improvable. The proposed algorithm avoids the sampling procedure, which usually consumes a heavy computational cost. Discussion of numerical and engineering examples in this work has demonstrated that the proposed algorithm provides an effective tool to deal with uncertainty analysis involving correlated inputs.     

催化裂化过程稳态优化控制研究

催化裂化装置是一个高度非线性、时变、长时延、强耦合、分布参数和不确定性的复杂 系统.在研究其过程机理的基础上,定义了一种模糊神经网络用以建模,用自相关函数检验法检 验模型的正确性,再用改进的Frank-Wolfe算法进行稳态优化计算,并以一炼油厂催化裂化装 置为对象进行试验,研究其辨识、建模和稳态优化控制.这种模糊神经网络具有隐层数多、隐层 结点数多、泛化能力和逼近能力强、收敛速度快的优点,更突出的特点还在于可由输出端对输入 求导,为稳态优化计算提供了极大方便,它与改进的Frank-Wolfe算法相结合用于解决非线性 复杂生产过程的建模和稳态优化控制问题是可行的.     

Genetic polynomial regression as input selection algorithm for non-linear identification

The performance of non-linear identification techniques is often determined by the appropriateness of the selected input variables and the corresponding time lags. High correlation coefficients between candidate input variables in addition to a non-linear relation with the output signal induce the need for an appropriate input selection methodology. This paper proposes a genetic polynomial regression technique to select the significant input variables for the identification of non-linear dynamic systems with multiple inputs. Statistical tools are presented to visualize and to process the results from different selection runs. The evolutionary approach can be used for a wide range of identification techniques and only requires a minimal input and a priori knowledge from the user. The evolutionary selection algorithm has been applied on a real-world example to illustrate its performance. The engine load in a combine harvester is highly variable in time and should be kept below an allowable limit during automatic ground speed control mode. The genetic regression process has been used to select those measurement variables that have a significant impact on the engine load and that will act as measurement variables of a non-linear model-based engine load controller.     

基于特征选择的网络入侵检测模型

研究网络安全问题,网络入侵手段多样,特征多,存在大量不利的冗余特征,传统网络入侵检测不考虑特征冗余,检测效率和正确论低。为更一步提高了网络安全,提出一种特征选择的网络入侵检测模模型。采用粒子群算法对网络系统状态特征和支持向量机参数进行同步选择,找到最优网络入侵检测模型特征和模型参数,降低了模型的输入样本维数。仿真结果表明,改进算法可降低特征维数,消除了不利于提高检测结果的冗余特征,并提高了网络入侵检测正确率,适合于小样本、实时要求高的网络入侵检测。     

An approximation to solve regression problems with a genetic fuzzy rule ordinal algorithm

Regression problems try estimating a continuous variable from a number of characteristics or predictors. Several proposals have been made for regression models based on the use of fuzzy rules; however, all these proposals make use of rule models in which the irrelevance of the input variables in relation to the variable to be approximated is not taken into account. Regression problems share with the ordinal classification the existence of an explicit relationship of order between the values of the variable to be predicted. In a recent paper, the authors have proposed an ordinal classification algorithm that takes into account the detection of the irrelevance of input variables. This algorithm extracts a set of fuzzy rules from an example set, using as the basic model a sequential covering strategy along with a genetic algorithm. In this paper, a proposal for a regression algorithm based on this ordinal classification algorithm is presented. The proposed model can be interpreted as a multiclassifier and multilevel system that learns at each stage using the knowledge gained in previous stages. Due to similarities between regression and ordinal problems as well as the use of a set of ordinal algorithms, an error interval can be returned with the regression output value. Experimental results show the good behavior of the proposal as well as the results of the error interval.     

An adaptive orthogonal search algorithm for model subset selection and non-linear system identification

A new adaptive orthogonal search (AOS) algorithm is proposed for model subset selection and non-linear system identification. Model structure detection is a key step in any system identification problem. This consists of selecting significant model terms from a redundant dictionary of candidate model terms, and determining the model complexity (model length or model size). The final objective is to produce a parsimonious model that can well capture the inherent dynamics of the underlying system. In the new AOS algorithm, a modified generalized cross-validation criterion, called the adjustable prediction error sum of squares (APRESS), is introduced and incorporated into a forward orthogonal search procedure. The main advantage of the new AOS algorithm is that the mechanism is simple and the implementation is direct and easy, and more importantly it can produce efficient model subsets for most non-linear identification problems.     

基于RBF神经网络辅助的自适应UKF算法研究

卡尔曼滤波能在测量噪声干扰下对系统状态进行无偏估计。但无论是扩展卡尔曼滤波（EKF）算法,还是无轨迹卡尔曼滤波（UKF）算法,都无法避免滤波发散现象。给出利用径向基函数(RBF)神经网络的自适应调整能力来对卡尔曼滤波输出进行校正,从而避免输出发散的算法。计算机模拟和实际应用表明,基于RBFNN的卡尔曼滤波算法可以有效防止输出发散。     

基于特征选择的轻量级入侵检测系统

基于特征选择的入侵检测系统处理的数据含有大量的冗余与噪音特征,使得系统耗用的计算资源很大,导致系统训练时间长、实时性差,检测效果不好.特征选择算法能够很好地消除冗余和噪音特征,为了提高入侵检测系统的检测速度和效果,对基于特征选择的入侵检测系统进行研究是必要的.综述了这一领域的研究进展,从过滤器、封装器、混合器3种模式对基于特征选择的轻量级入侵检测系统进行分类比较,分析和总结各种系统的优缺点以及它们各自适用的条件,最后指出入侵检测领域特征选择的发展趋势.特征选择不仅可以提升入侵检测系统的性能,而且使得对入侵检测的研究向特征提取算法的方向转移.     

A new redundant variable pruning approach--minor latent variable perturbation-PLS used for QSAR studies on anti-HIV drugs

A new approach for eliminating the redundant variables in the multivariable data matrix encountered in QSAR studies, minor latent variable perturbation (MLVP)-PLS method has been proposed. In the latent variable (LV) space, the minor latent variables (LVs) with small covariances are mainly formulated by linear combinations of the redundant variables including information-deficient and highly correlative ones, while the major LVs with large covariances are mainly contributed by the informative variables. Deleting a minor LV, which is equivalent to a perturbation for LV space, could make the redundant variables not well be represented in LV subspace, leading to strong variation of their PLS regression coefficients. The informative variables could still be normally represented in LV subspace with the PLS regression coefficients remaining relatively stable. MLVP-PLS utilizes this fact to discriminate the informative and redundant variables. It gradually identifies and eliminates the redundant variables according to the relative variation of PLS regression coefficients after perturbations are given. The elimination process is terminated according to some proposed criteria. Applying the method to the quantitative structure-activity relationship (QSAR) studies on TIBO derivatives as potential anti-HIV drugs has demonstrated the feasibility and robustness of the proposed approach. A deeper insight into the effect of different structural parameters on the bio-activity of TIBO derivatives has been reached.     

Semi-supervised relevance index for feature selection

This paper presents a new relevance index based on mutual information that is based on labeled and unlabeled data. The proposed index, which is based in Mutual Information, takes into account the similarity between features and their joint influence on the output variable. Based on this principle, a method to select features is developed to eliminate redundant and irrelevant features when the relevance index value is less then a threshold value. A strategy to set the threshold is also proposed in this work. Experiments show that the new method is capable of capturing important joint relations between input and output variables, which are incorporated into a new feature selection clustering approach.