Information recommendation using ICA

We describe an information filtering system using independent component analysis (ICA). A document–word matrix is generally sparse and has an ambiguity of synonyms. To solve this problem, we propose a method to use document vectors represented by independent components. An independent component generated by ICA is considered as a topic. In practice, we map the document vectors into a topics space. Since some independent components are useless for recommendation, we select the necessary components from all independent components by a maximum distance algorithm (MDA). Although Euclidean distance is usually used by MDA, we propose topic selection by cosine-distance-based MDA to solve the mismatch of similarities in information filtering. We create a user profile from the transformed data with a genetic algorithm (GA). Finally, we recommend documents with the user profile and evaluate the accuracy by imputation precision. We have carried out an evaluation experiment to confirm the practicality of the proposed method.This work was presented, in part, at the 9th International Symposium on Artificial Life and Robotics, Oita, Japan, January 28–30, 2004     

Model selection for convolutive ICA with an application to spatiotemporal analysis of EEG

We present a new algorithm for maximum likelihood convolutive independent component analysis (ICA) in which components are unmixed using stable autoregressive filters determined implicitly by estimating a convolutive model of the mixing process. By introducing a convolutive mixing model for the components, we show how the order of the filters in the model can be correctly detected using Bayesian model selection. We demonstrate a framework for deconvolving a subspace of independent components in electroencephalography (EEG). Initial results suggest that in some cases, convolutive mixing may be a more realistic model for EEG signals than the instantaneous ICA model.     

Fast and robust fixed-point algorithms for independent componentanalysis

Independent component analysis (ICA) is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible. We use a combination of two different approaches for linear ICA: Comon's information theoretic approach and the projection pursuit approach. Using maximum entropy approximations of differential entropy, we introduce a family of new contrast functions for ICA. These contrast functions enable both the estimation of the whole decomposition by minimizing mutual information, and estimation of individual independent components as projection pursuit directions. The statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model, and it is shown how to choose contrast functions that are robust and/or of minimum variance. Finally, we introduce simple fixed-point algorithms for practical optimization of the contrast functions     

Feature Extraction Using Independent Components of Each Category

We describe an application of independent component analysis (ICA) to pattern recognition in order to evaluate the effectiveness of features extracted by ICA. We propose a recognition method suitable for independent components that consists of modules for each category. A module has two parts: feature extraction and classification. Features are independent components estimated by ICA and outputs of modules are candidates for categories. These candidates are combined and categories are decided with a majority rule. This recognition method is applied to two tasks: hand-written digits in the MNIST database and acoustic diagnosis for a compressor as real-world tasks. A FastICA algorithm is applied to extracting independent features in the proposed method. Through recognition experiments, we demonstrate that the ICA of each category extracts useful features for these tasks and the independent components are superior to the principal components in recognition accuracy. Manabu Kotani - Deceased     

独立成分分析在表情识别中的应用

独立成分分析(ICA)是一种基于信号高阶统计特性的分析方法,本文尝试将这种方法应用于人脸表情的特征提取。首先对预处理后的图像用FastICA算法计算得到解混矩阵以及此训练样本集的影像独立基成分,然后利用影像独立基来构造一个投影空间,最后利用待识别的表情图像在这个空间上作空间影射,所得到得投影系数用以实现分类。为了减少运算量,本文研究了降维的训练样本集的独立成分分析。     

基于PCA/ICA的多数据流关联及模式发现

基于主成分分析技术、独立分量分析技术以及多数据流模型,将用于数据和信号分析的PCA/ICA方法应用于多数据流模型,提出多数据流关联度分析和模式发现的新模型。该模型适用于解决在线混合数据流分离,对挖掘多数据流潜在独立内因有良好效果。探讨模型的健壮性和实时性,并在实验中验证了系统性能。     

Statistical process monitoring with independent component analysis

In this paper we propose a new statistical method for process monitoring that uses independent component analysis (ICA). ICA is a recently developed method in which the goal is to decompose observed data into linear combinations of statistically independent components [1 and 2]. Such a representation has been shown to capture the essential structure of the data in many applications, including signal separation and feature extraction. The basic idea of our approach is to use ICA to extract the essential independent components that drive a process and to combine them with process monitoring techniques. I², I_e² and SPE charts are proposed as on-line monitoring charts and contribution plots of these statistical quantities are also considered for fault identification. The proposed monitoring method was applied to fault detection and identification in both a simple multivariate process and the simulation benchmark of the biological wastewater treatment process, which is characterized by a variety of fault sources with non-Gaussian characteristics. The simulation results clearly show the power and advantages of ICA monitoring in comparison to PCA monitoring.     

基于独立成分分析及其扩展模型的工业过程监测方法综述

独立成分分析(independent component analysis,ICA)是一种多变量统计分析方法,常用于非高斯过程监测,它能够有效利用信号的高阶统计信息(三阶以上)提取相互独立的独立成分,在工业过程监测中得到了广泛的应用,是当前国际过程监测领域的研究热点.鉴于此,介绍经典ICA模型、改进ICA模型及其在工业...     

独立成分相关分析的自适应故障监测方法

工业过程数据具有动态、非高斯等特性.独立成分分析(independent component analysis, ICA)既可以分析数据的非高斯形式,又可以极大地去除多变量间的耦合且满足独立性要求.本文引入粒子群算法优化ICA模型参数,自适应地确定独立成分个数.同时,提出一种基于隐马尔科夫链模型(hidden Markov model, HMM)的自适应检测限设计方法,将时间相关数据块的特征信息变化作为过程故障的检测依据.首先利用由时间窗方法确定的独立成分组成监测矩阵来训练HMM模型,旨在提高独立成分间相关性水平的表示能力;然后将得到的HMM模型对监测矩阵进行相关性评估,并在一定容许裕度的基础上设计评估值的自适应因子及检测限,并据此监测特征信息变化,动态地进行在线故障检测.最后, Tennessee Eastman (TE)仿真平台的实验结果表明了所提方法的有效性.     

Applications of ICA for the enhancement and classification of polarimetric SAR images

Independent components analysis (ICA) based methods for polarimetric synthetic aperture radar (SAR) image speckle reduction and ground object classification are studied. Several independent components can be extracted from polarimetric SAR images using ICA directly. The component with lowest speckle index is regarded as the scene after speckle reduction. The disadvantage of this method is that only one image is kept and most polarization information will be lost. In this paper, we use ICA‐sparse‐coding shrinkage (ICA‐SPS) based speckle reduction method, which is implemented on each individual image and can keep polarization information. It is carried out on the combined channels obtained by Pauli‐decomposition rather than original polarization channels in order to keep relative phase information among polarization channels and get better performance. After ICA‐SPS, the effect of speckle suppression on SAR image classification can be compared favourably with other methods by combining the channels into a false colour image. At last, a new ICA‐based classification method is presented. In this method, four independent components are separated by ICA from five polarization and combined channels. One of these independent components which includes little ground object information is regarded as speckle noise and therefore be discarded. The remaining three components can be treated as subordination coefficients of three kinds of targets. A classified image can be obtained based on the components. And by composing these three channels in RGB colour pattern, a false colour image can be constructed.     

基于改进的独立分量分析的人脸识别方法

将独立分量分析（Independent Component Analysis，ICA）作为人脸特征提取方法。ICA所提取的特征分类能力强、相互独立，对像素间高阶统计特性敏感，并且不易受光照变化的影响。实验结果表明，基于IcA的人脸特征提取方法的识别性能优于特征脸法。针对传统的ICA算法（Informax算法）存在迭代次数多，难收敛，并且需要人工设定步长来调整学习速度的不足，本文采用FastICA作为ICA的快速算法，并将其关键迭代步骤加以改进，减少了耗时的雅可比矩阵求逆的运算次数。所提出的改进的FastICA具有无需人工参与，收敛速度快，迭代次数少的优点。在特征选择方面，本文将遗传算法（Genetie Algorithm，GA）应用到独立分量的选择与优化中，从而在保证较高识别性能的前提下，获得最优的人脸特征子集。     

Lattice independent component analysis for functional magnetic resonance imaging

We introduce a lattice independent component analysis (LICA) unsupervised scheme to functional magnetic resonance imaging (fMRI) data analysis. LICA is a non-linear alternative to independent component analysis (ICA), such that ICA’s statistical independent sources correspond to LICA’s lattice independent sources. In this paper, LICA uses an incremental lattice source induction algorithm (ILSIA) to induce the lattice independent sources from the input dataset. The ILSIA computes a set of Strongly Lattice Independent vectors using properties of lattice associative memories regarding Lattice Independence and Chebyshev best approximation. The lattice independent sources constitute a set of Affine Independent vectors that define a simplex covering the input data. LICA carries out data linear unmixing based on the lattice independent sources basis. Therefore, LICA is a hybrid combination of a non-linear lattice based component and a linear unmixing component. The principal advantage over ICA is that LICA does not impose any probabilistic model assumptions on the data sources. We compare LICA with ICA in two case studies. Firstly, including simulated fMRI data, LICA discovers the spatial location of meaningful sources with less ambiguity than ICA. Secondly, including real data from an auditory stimulation experiment, LICA improves over some state of the art ICA variants discovering the activation patterns detected by Statistical Parametric Mapping (SPM) on the same data.     

基于正交独立成分分析的过程数据建模

针对非高斯数据分布过程中回归预测精度不足的问题,提出一种在独立成分分析（ICA）的基础上与正交信号校正（OSC）相结合的多元线性回归（MLR）方法——正交独立成分回归（O-ICR）．首先将原输入数据通过正交ICA（O-ICA）进行预处理,去除ICA在提取高阶统计量时带来的与Y无关的干扰变化,然后对校正后的X提取独立成分,代替原输入数据建立与Y之间的回归预测模型．与传统的ICR相比,该方法提取的独立成分经过校正可使回归模型的预测精度更高．最后通过Tennessee Eastman（TE）过程的质量预测仿真,验证了该建模方法的有效性．     

Independent component analysis: A flexible nonlinearity and decorrelating manifold approach

Independent component analysis (ICA) finds a linear transformation to variables that are maximally statistically independent. We examine ICA and algorithms for finding the best transformation from the point of view of maximizing the likelihood of the data. In particular, we discuss the way in which scaling of the unmixing matrix permits a "static" nonlinearity to adapt to various marginal densities. We demonstrate a new algorithm that uses generalized exponential functions to model the marginal densities and is able to separate densities with light tails. We characterize the manifold of decorrelating matrices and show that it lies along the ridges of high-likelihood unmixing matrices in the space of all unmixing matrices. We show how to find the optimum ICA matrix on the manifold of decorrelating matrices, and as an example we use the algorithm to find independent component basis vectors for an ensemble of portraits.     

Independent component analysis using Potts models

We explore the extending application of Potts encoding to the task of independent component analysis, which primarily deals with the problem of minimizing the Kullback-Leibler divergence between the joint distribution and the product of all marginal distributions of output components. The competitive mechanism of Potts neurons is used to encode the overlapping projections from observations to output components. Based on these projections, the marginal distributions and the entropy of output components are made tractable for computation and the adaptation of the de-mixing matrix toward independent output components is obtained. The Potts model for ICA is well formulated by an objective function subject to a set of constraints, which leads to a novel energy function. A hybrid of the mean field annealing and the gradient descent method is applied to the energy function. Our approach to independent component analysis presents a new criterion for ICA. The performance of the Potts model for ICA given by our numerical simulations is encouraging.     

A common neural-network model for unsupervised exploratory data analysis and independent component analysis.

This paper presents the derivation of an unsupervised learning algorithm, which enables the identification and visualization of latent structure within ensembles of high-dimensional data. This provides a linear projection of the data onto a lower dimensional subspace to identify the characteristic structure of the observations independent latent causes. The algorithm is shown to be a very promising tool for unsupervised exploratory data analysis and data visualization. Experimental results confirm the attractiveness of this technique for exploratory data analysis and an empirical comparison is made with the recently proposed generative topographic mapping (GTM) and standard principal component analysis (PCA). Based on standard probability density models a generic nonlinearity is developed which allows both (1) identification and visualization of dichotomised clusters inherent in the observed data and (2) separation of sources with arbitrary distributions from mixtures, whose dimensionality may be greater than that of number of sources. The resulting algorithm is therefore also a generalized neural approach to independent component analysis (ICA) and it is considered to be a promising method for analysis of real-world data that will consist of sub- and super-Gaussian components such as biomedical signals.     

有限制的MFA-ICA的算法及其在图像特征提取中的应用

统的独立成分分析(IndependentComponentAnalysis,ICA)是一种无噪声模型,而实际应用中噪声是存在的。根据多元统计中的因子分析模型,改变其假设条件,从而得到一种有噪声ICA模型,对于模型参数,引入平均场近似(MeanFieldApproximation,MFA)原理来求解。针对图像特征提取,通过增加对模型参数的一些限制,使其能得到更为独立的图像特征,为图像识别提供更可靠的特征信息,从而大大提高识别率。通过仿真模拟图形以及ORL人脸数据进行实验,将传统的独立成分分析算法、无限制的MFA ICA算法以及增加限制条件的MFA ICA算法进行比较,从仿真模拟图形实验结果看,限制的MFA ICA算法能分离出更独立的特征,同时利用限制的MFA ICA算法识别效果明显优于传统ICA算法和无限制MFA ICA算法。     

基于逆运动学和重构式ICA的人体运动风格分析与合成

使用独立成分分析（Independent component analysis,ICA）来建模运动风格、合成风格化的人体运动,是一种有效且有前景的手段.为了避免现有方法在设定独立成分个数或子空间结构时的人为影响,并提高风格成分的质量,提出一种基于重构式独立成分分析的运动风格分析方法.由于放弃了混合矩阵的正交性约束,一方面,拥有了更多的自由度来表示各独立成分;另一方面,利用特征的过完备性以及自身在特征选择时的稀疏特性,能够自动地确立独立成分数目.此外,通过结合基于主测地线分析的逆运动学与运动过渡技术,该方法能够合成包含多种风格、任意长度的行走运动,同时还能通过编辑特定帧的人体姿势来约束合成的结果.实验结果表明,该方法能够有效地分析出行走、跳跃和踢腿等运动中代表风格的独立成分,并根据用户对风格的编辑,实时地生成自然、平滑的运动.     

Input variable selection: mutual information and linear mixing measures

Determining the most appropriate inputs to a model has a significant impact on the performance of the model and associated algorithms for classification, prediction, and data analysis. Previously, we proposed an algorithm ICAIVS which utilizes independent component analysis (ICA) as a preprocessing stage to overcome issues of dependencies between inputs, before the data being passed through to an input variable selection (IVS) stage. While we demonstrated previously with artificial data that ICA can prevent an overestimation of necessary input variables, we show here that mixing between input variables is common in real-world data sets so that ICA preprocessing is useful in practice. This experimental test is based on new measures introduced in this paper. Furthermore, we extend the implementation of our variable selection scheme to a statistical dependency test based on mutual information and test several algorithms on Gaussian and sub-Gaussian signals. Specifically, we propose a novel method of quantifying linear dependencies using ICA estimates of mixing matrices with a new linear mixing measure (LMM).     

结合纯像元提取和ICA的高光谱降维方法*

摘要：为了实现高光谱降维并保留重要的光谱特征,通过独立分量分析（Independent Component Analysis, ICA）混合模型和高光谱线性模型的对比分析,提出了结合纯像元提取和ICA的高光谱数据降维方法。该方法通过估计虚拟维数(Virtual Dimensionality, VD)确定特征个数,采用自动目标生成过程（Automatic Target Generation Process, ATGP）从原始数据中提取纯像元向量,作为ICA算法的初始化向量,以负熵为目标函数产生独立分量,并通过高阶统计量筛选实现高光谱数据的降维。分类实验结果表明,该方法不仅解决了传统ICA的随机排序问题,而且与经典降维算法主分量分析（Principal Components Analysis, PCA）相比,分类精度提高了6.83%,在大大降低高光谱数据量的情况下很好的保留了高光谱数据的特征,有利于数据的后续分析和应用。