Information filtering using SVD and ICA

We propose an information filtering system for documents by a user profile using latent semantics obtained by singular value decomposition (SVD) and independent component analysis (ICA). In information filtering systems, it is useful to analyze the latent semantics of documents. ICA is one method to analyze the latent semantics. We assume that topics are independent of each other. Hence, when ICA is applied to documents, we obtain the topics included in the documents. By using SVD remove noises before applying ICA, we can improve the accuracy of topic extraction. By representation of the documents with those topics, we improve recommendations by the user profile. In addition, we construct a user profile with a genetic algorithm (GA) and evaluate it by 11-point average precision. We carried out an experiment using a test collection to confirm the advantages of the proposed method. This work was presented in part at the 10th International Symposium on Artificial Life and Robotics, Oita, Japan, February 4–6, 2005     

Information filtering using a probabilistic model

We propose an information filtering system based on a probabilistic model. We make an assumption that a document consists of words which occur according to a probability distribution, and regard a document as a sample drawn according to that distribution. In this article, we adopt a multinomial distribution and represent a document as probability which has random values as the words in the document. When an information filtering system selects information, it uses the similarity between the user's interests (a user profile) and a document. Since our proposed system is constructed under the probabilistic model, the similarity is defined using the Kullback Leibler divergence. To create the user profile, we must optimize the Kullback Leibler divergence. Since the Kullback Leibler divergence is a nonlinear function, we use a genetic algorithm to optimize it. We carry out experiments and confirm effectiveness of the proposed method. This work was presented in part at the 10th International Symposium on Artificial Life and Robotics, Oita, Japan, February 4–6, 2005     

Feature Extraction Using Independent Components of Each Category

We describe an application of independent component analysis (ICA) to pattern recognition in order to evaluate the effectiveness of features extracted by ICA. We propose a recognition method suitable for independent components that consists of modules for each category. A module has two parts: feature extraction and classification. Features are independent components estimated by ICA and outputs of modules are candidates for categories. These candidates are combined and categories are decided with a majority rule. This recognition method is applied to two tasks: hand-written digits in the MNIST database and acoustic diagnosis for a compressor as real-world tasks. A FastICA algorithm is applied to extracting independent features in the proposed method. Through recognition experiments, we demonstrate that the ICA of each category extracts useful features for these tasks and the independent components are superior to the principal components in recognition accuracy. Manabu Kotani - Deceased     

基于聚类分析策略的用户偏好挖掘

利用训练文档集准确高效地挖掘隐藏的用户文本偏好和概念向量是文本信息过滤和多文档自动文摘等自然语言处理应用的关键技术之一。针对训练文本集中往往存在多个主题类别的问题,提出一种基于聚类分析策略的文本偏好挖掘方法。其基本思路是对训练文档集进行聚类处理,然后对同主题文档进行共性分析,并经过特征权值调整和特征约简,获得表示用户不同主题偏好的概念向量。实验结果表明该方法具有对用户的文本偏好刻画更加精确,对相关阈值变化不敏感等优点,可以与Rocchio等算法结合来进行用户兴趣建模。     

Independent component analysis in the blind watermarking of digital images

We propose a new method for the blind robust watermarking of digital images based on independent component analysis (ICA). We apply ICA to compute some statistically independent transform coefficients where we embed the watermark. The main advantages of this approach are twofold. On the one hand, each user can define its own ICA-based transformation. These transformations behave as “private-keys” of the method. On the other hand, we will show that some of these transform coefficients have white noise-like spectral properties. We develop an orthogonal watermark to blindly detect it with a simple matched filter. We also address some relevant issues as the perceptual masking of the watermark and the estimation of the detection probability. Finally, some experiments have been included to illustrate the robustness of the method to common attacks and to compare its performance to other transform domain watermarking algorithms.     

Construction of a classifier using AdaBoost for information filtering

We describe an information filtering system using AdaBoost. To realize the filtering system, we created a user profile which presents the users interests. Since the users interests are complex, the user profile becomes a nonlinear discriminant function. However, it is difficult to decide on an appropriate discriminant function. We used AdaBoost to modify the appropriate user profile. AdaBoost is an ensemble algorithm which combines weak learners and improves the accuracy of a classifier. In this method, the weak learners for AdaBoost is a linear discriminant function which is created with a genetic algorithm. We carried out experiments for an information filtering service on an NTCIR2 test collection, and we discuss the effectiveness of the method.This work was presented, in part, at the 9th International Symposium on Artificial Life and Robotics, Oita, Japan, January 28–30, 2004     

Independent component analysis: A flexible nonlinearity and decorrelating manifold approach

Independent component analysis (ICA) finds a linear transformation to variables that are maximally statistically independent. We examine ICA and algorithms for finding the best transformation from the point of view of maximizing the likelihood of the data. In particular, we discuss the way in which scaling of the unmixing matrix permits a "static" nonlinearity to adapt to various marginal densities. We demonstrate a new algorithm that uses generalized exponential functions to model the marginal densities and is able to separate densities with light tails. We characterize the manifold of decorrelating matrices and show that it lies along the ridges of high-likelihood unmixing matrices in the space of all unmixing matrices. We show how to find the optimum ICA matrix on the manifold of decorrelating matrices, and as an example we use the algorithm to find independent component basis vectors for an ensemble of portraits.     

Lattice independent component analysis for functional magnetic resonance imaging

We introduce a lattice independent component analysis (LICA) unsupervised scheme to functional magnetic resonance imaging (fMRI) data analysis. LICA is a non-linear alternative to independent component analysis (ICA), such that ICA’s statistical independent sources correspond to LICA’s lattice independent sources. In this paper, LICA uses an incremental lattice source induction algorithm (ILSIA) to induce the lattice independent sources from the input dataset. The ILSIA computes a set of Strongly Lattice Independent vectors using properties of lattice associative memories regarding Lattice Independence and Chebyshev best approximation. The lattice independent sources constitute a set of Affine Independent vectors that define a simplex covering the input data. LICA carries out data linear unmixing based on the lattice independent sources basis. Therefore, LICA is a hybrid combination of a non-linear lattice based component and a linear unmixing component. The principal advantage over ICA is that LICA does not impose any probabilistic model assumptions on the data sources. We compare LICA with ICA in two case studies. Firstly, including simulated fMRI data, LICA discovers the spatial location of meaningful sources with less ambiguity than ICA. Secondly, including real data from an auditory stimulation experiment, LICA improves over some state of the art ICA variants discovering the activation patterns detected by Statistical Parametric Mapping (SPM) on the same data.     

Edge detection algorithm based on ICA-domain shrinkage in noisy images

We propose a robust edge detection method based on ICA-domain shrinkage （in- dependent component analysis）. It is known that most basis functions extracted from natural images by ICA are sparse and similar to localized and oriented receptive fields, and in the proposed edge detection method, a target image is first transformed by ICA basis functions and then the edges are detected or recon- structed with sparse components. Furthermore, by applying a shrinkage algorithm to filter out the components of noise in ICA-domain, we can readily obtain the sparse components of the original image, resulting in a kind of robust edge detec- tion even for a noisy image with a very low SN ratio. The efficiency of the proposed method is demonstrated by experiments with some natural images.     

Query-Free News Search

Many daily activities present information in the form of a stream of text, and often people can benefit from additional information on the topic discussed. TV broadcast news can be treated as one such stream of text; in this paper we discuss finding news articles on the web that are relevant to news currently being broadcast. We evaluated a variety of algorithms for this problem, looking at the impact of inverse document frequency, stemming, compounds, history, and query length on the relevance and coverage of news articles returned in real time during a broadcast. We also evaluated several postprocessing techniques for improving the precision, including reranking using additional terms, reranking by document similarity, and filtering on document similarity. For the best algorithm, 84–91% of the articles found were relevant, with at least 64% of the articles being on the exact topic of the broadcast. In addition, a relevant article was found for at least 70% of the topics.     

融合用户特征优化聚类的协同过滤算法

针对推荐系统领域中应用最广泛的协同过滤推荐算法仍伴随着数据稀疏性、冷启动和扩展性问题,基于用户冷启动和扩展性问题,提出了基于改进聚类的PCEDS(pearson correlation coefficient and euclidean distance similarity)协同过滤推荐算法。首先针对用户属性特征,采用优化的K-means聚类算法对其聚类,然后结合基于信任度的用户属性特征相似度模型和用户偏好相似度模型,形成一种新颖的PCEDS相似度模型,对聚类结果建立预测模型。实验结果表明:提出的PCEDS算法比传统的协同过滤推荐算法在均方根误差(RMSE)上降低5%左右,并且推荐准确率(precision)和召回率(recall)均有明显提高,缓解了冷启动问题,同时聚类技术可以节省系统内存计算空间,从而提高了推荐效率。     

Ensemble learning for independent component analysis

It is well known that the applicability of independent component analysis (ICA) to high-dimensional pattern recognition tasks such as face recognition often suffers from two problems. One is the small sample size problem. The other is the choice of basis functions (or independent components). Both problems make ICA classifier unstable and biased. In this paper, we propose an enhanced ICA algorithm by ensemble learning approach, named as random independent subspace (RIS), to deal with the two problems. Firstly, we use the random resampling technique to generate some low dimensional feature subspaces, and one classifier is constructed in each feature subspace. Then these classifiers are combined into an ensemble classifier using a final decision rule. Extensive experimentations performed on the FERET database suggest that the proposed method can improve the performance of ICA classifier.     

基于Mahout的新用户推荐算法的设计与实现

为了解决大数据背景下新用户因没有历史数据而导致推荐难和推荐效率低等问题,提出将基于Mahout的协同过滤算法与基于MapReduce的Top N算法相结合的技术方法,来实现新用户推荐算法,从而构建新用户推荐系统的架构,并对Hadoop Top N算法以及Mahout中协同过滤算法进行设计与实现。理论分析和实验验证表明,该新用户推荐算法在推荐效率、对大规模数据处理的伸缩性以及推荐质量上都明显优于单独使用协同过滤算法的新用户推荐。     

组合局部多通道Gabor滤波器和ICA的人脸描述与识别

针对人脸识别是当前人工智能和模式识别的研究热点,提出了一种组合局部Gabor滤波器组和ICA技术(简称LMGICA)的人脸描述方法,首先对归一化的人脸图像进行采样分块,然后对局部子块进行多方向、多分辨率Gabor小波滤波,并提取其对应不同方向、不同尺度的多个Gabor 幅值域图谱(local Gabor magnitude map,LGMM),接着由滤波图像直接构建高维特征矢量;再将这些高维特征矢量通过主成分分析进行降维;最后采用ICA技术分析和提取降维后的特征矢量中的独立成分用于识别分类。通过与经典Ga     

Face recognition using IPCA-ICA algorithm

In this paper, a fast incremental principal non-Gaussian directions analysis algorithm, called IPCA-ICA, is introduced. This algorithm computes the principal components of a sequence of image vectors incrementally without estimating the covariance matrix (so covariance-free) and at the same time transforming these principal components to the independent directions that maximize the non-Gaussianity of the source. Two major techniques are used sequentially in a real-time fashion in order to obtain the most efficient and independent components that describe a whole set of human faces database. This procedure is done by merging the runs of two algorithms based on principal component analysis (PCA) and independent component analysis (ICA) running sequentially. This algorithm is applied to face recognition problem. Simulation results on different databases showed high average success rate of this algorithm compared to others.     

Single-Mixture Audio Source Separation by Subspace Decomposition of Hilbert Spectrum

A novel technique is developed to separate the audio sources from a single mixture. The method is based on decomposing the Hilbert spectrum (HS) of the mixed signal into independent source subspaces. Hilbert transform combined with empirical mode decomposition (EMD) constitutes HS, which is a fine-resolution time-frequency representation of a nonstationary signal. The EMD represents any time-domain signal as the sum of a finite set of oscillatory components called intrinsic mode functions (IMFs). After computing the spectral projections between the mixed signal and the individual IMF components, the projection vectors are used to derive a set of spectral independent bases by applying principal component analysis (PCA) and independent component analysis (ICA). A k-means clustering algorithm based on Kulback-Leibler divergence (KLd) is introduced to group the independent basis vectors into the number of component sources inside the mixture. The HS of the mixed signal is projected onto the space spanned by each group of basis vectors yielding the independent source subspaces. The time-domain source signals are reconstructed by applying the inverse transformation. Experimental results show that the proposed algorithm performs separation of speech and interfering sound from a single mixture     

Model selection for convolutive ICA with an application to spatiotemporal analysis of EEG

We present a new algorithm for maximum likelihood convolutive independent component analysis (ICA) in which components are unmixed using stable autoregressive filters determined implicitly by estimating a convolutive model of the mixing process. By introducing a convolutive mixing model for the components, we show how the order of the filters in the model can be correctly detected using Bayesian model selection. We demonstrate a framework for deconvolving a subspace of independent components in electroencephalography (EEG). Initial results suggest that in some cases, convolutive mixing may be a more realistic model for EEG signals than the instantaneous ICA model.     

基于最短路径和自然梯度的过完备ICA算法

独立成分分析(ICA)是一种在给出的随机向量中找出统计独立的数据的统计方法，而过完备独立成分分析则是ICA问题中的一类特殊的情形，它要的源信号的数目比观测信号的数目要多。该文提出了一种基于最短路径算法和自然梯度的解决过完备独立成分分析的新算法Turbo-overcomplete。该算法采用了最短路径方法来推断源信号和采用自然梯度的方法来学习基向量，并采用Turbo-overcomplete算法来进行语音信号分离的实验，并把实验结果与现在的一些过完备独立成份分析算法进行了比较。     

Acoustic signal separation of mechanical devices using ICA with band-pass filters

We propose a method of separating the acoustic signals of motors and the gears of mechanical devices by using independent component analysis (ICA) with band-pass filters. The frequency distribution of a recorded acoustic signal from the operating mechanical device can be divided into three fields, the low-frequency field, which corresponds to the frequency characteristics of the gear, the medium-frequency field, which is mixed with the frequency characteristics of the gear and the motor, and the high-frequency field, which corresponds to the frequency characteristics of the motor. Since only the medium-frequency components are a mixture of the acoustic signals of gears and motors, ICA with band-pass filters is expected to separate the acoustic signals of motors and gears more accurately than conventional ICA. The simulation and experimental results show that the proposed method can separate the acoustic signals of motors and gears of mechanical devices successfully.This work was presented, in part, at the 9th International Symposium on Artificial Life and Robotics, Oita, Japan, January 28–30, 2004     

基于Overcomplet ICA的声音压缩模型

独立成分分析(ICA)方法是近几年发展起来的一种新统计方法,旨在将所观测到的多维随机向量转换成统计上尽可能独立的成分。本文基于Overcomplete(过完备)ICA算法(SCO),提出了一种新的声音压缩模型。我们的实验实现了SCO的混合压缩与分离解压功能。