Bayesian hybrid generative discriminative learning based on finite Liouville mixture models

Recently hybrid generative discriminative approaches have emerged as an efficient knowledge representation and data classification engine. However, little attention has been devoted to the modeling and classification of non-Gaussian and especially proportional vectors. Our main goal, in this paper, is to discover the true structure of this kind of data by building probabilistic kernels from generative mixture models based on Liouville family, from which we develop the Beta-Liouville distribution, and which includes the well-known Dirichlet as a special case. The Beta-Liouville has a more general covariance structure than the Dirichlet which makes it more practical and useful. Our learning technique is based on a principled purely Bayesian approach which resulted models are used to generate support vector machine (SVM) probabilistic kernels based on information divergence. In particular, we show the existence of closed-form expressions of the Kullback-Leibler and Rényi divergences between two Beta-Liouville distributions and then between two Dirichlet distributions as a special case. Through extensive simulations and a number of experiments involving synthetic data, visual scenes and texture images classification, we demonstrate the effectiveness of the proposed approaches.     

Bayesian analysis of finite mixtures of multinomial and negative-multinomial distributions

The Bayesian implementation of finite mixtures of distributions has been an area of considerable interest within the literature. Computational advances on approximation techniques such as Markov chain Monte Carlo (MCMC) methods have been a keystone to Bayesian analysis of mixture models. This paper deals with the Bayesian analysis of finite mixtures of two particular types of multidimensional distributions: the multinomial and the negative-multinomial ones. A unified framework addressing the main topics in a Bayesian analysis is developed for the case with a known number of component distributions. In particular, theoretical results and algorithms to solve the label-switching problem are provided. An illustrative example is presented to show that the proposed techniques are easily applied in practice.     

Bayesian change detection based on spatial sampling and Gaussian mixture model

A Gaussian mixture model (GMM) and Bayesian inferencing based unsupervised change detection algorithm is proposed to achieve change detection on the difference image computed from satellite images of the same scene acquired at different time instances. Each pixel of the difference image is represented by a feature vector constructed from the difference image values of the neighbouring pixels to consider the contextual information. The feature vectors of the difference image are modelled as a GMM. The conditional posterior probabilities of changed and unchanged pixel classes are automatically estimated by partitioning GMM into two distributions by minimizing an objective function. Bayesian inferencing is then employed to segment the difference image into changed and unchanged classes by using the conditional posterior probability of each class. Change detection results are shown on real datasets.     

Infinite Liouville mixture models with application to text and texture categorization

This paper addresses the problem of proportional data modeling and clustering using mixture models, a problem of great interest and of importance for many practical pattern recognition, image processing, data mining and computer vision applications. Finite mixture models are broadly applicable to clustering problems. But, they involve the challenging problem of the selection of the number of clusters which requires a certain trade-off. The number of clusters must be sufficient to provide the discriminating capability between clusters required for a given application. Indeed, if too many clusters are employed overfitting problems may occur and if few are used we have a problem of underfitting. Here we approach the problem of modeling and clustering proportional data using infinite mixtures which have been shown to be an efficient alternative to finite mixtures by overcoming the concern regarding the selection of the optimal number of mixture components. In particular, we propose and discuss the consideration of infinite Liouville mixture model whose parameter values are fitted to the data through a principled Bayesian algorithm that we have developed and which allows uncertainty in the number of mixture components. Our experimental evaluation involves two challenging applications namely text classification and texture discrimination, and suggests that the proposed approach can be an excellent choice for proportional data modeling.     

On Bayesian analysis of a finite generalized Dirichlet mixture via a Metropolis-within-Gibbs sampling

In this paper, we present a fully Bayesian approach for generalized Dirichlet mixtures estimation and selection. The estimation of the parameters is based on the Monte Carlo simulation technique of Gibbs sampling mixed with a Metropolis-Hastings step. Also, we obtain a posterior distribution which is conjugate to a generalized Dirichlet likelihood. For the selection of the number of clusters, we used the integrated likelihood. The performance of our Bayesian algorithm is tested and compared with the maximum likelihood approach by the classification of several synthetic and real data sets. The generalized Dirichlet mixture is also applied to the problems of IR eye modeling and introduced as a probabilistic kernel for Support Vector Machines.

A Bayesian approach to model dispersal for decision support

In agricultural and environmental sciences dispersal models are often used for risk assessment to predict the risk associated with a given configuration and also to test scenarios that are likely to minimise those risks. Like any biological process, dispersal is subject to biological, climatic and environmental variability and its prediction relies on models and parameter values which can only approximate the real processes. In this paper, we present a Bayesian method to model dispersal using spatial configuration and climatic data (distances between emitters and receptors; main wind direction) while accounting for uncertainty, with an application to the prediction of adventitious presence rate of genetically modified maize (GM) in a non-GM field. This method includes the design of candidate models, their calibration, selection and evaluation on an independent dataset. A group of models was identified that is sufficiently robust to be used for prediction purpose. The group of models allows to include local information and it reflects reliably enough the observed variability in the data so that probabilistic model predictions can be performed and used to quantify risk under different scenarios or derive optimal sampling schemes.     

基于概率算法自适应更新背景的运动车辆检测

交通流量检测是智能交通系统中的一个重要研究方向和热点问题,基于视频的车辆检测是交通流量采集分析的核心技术,它为交通流量参数的实时获取提供了可能。为实现在复杂交通视频场景中实时准确检测各类的运动车辆,在研究传统背景差分算法的缺点的工作基础上,提出一个自适应的贝叶斯概率背景检测算法,进而完成了较准确的运动车辆分类检测。实验结果表明该方法具有高效实时的特点,能够较准确地实现复杂交通路面的背景提取和运动车辆的检测,具有良好的鲁棒性。     

基于贝叶斯推理的多线索视觉注意模型

针对大多数视觉注意模型都采用简单加权线性融合的方式获取显著图,提出了一个更符合生物学机制的基于贝叶斯推理的多线索视觉注意模型,模拟视觉系统腹侧通路与背侧通路中的视觉注意过程,采用贝叶斯推理的方式集成自顶向下与自底向上的信息,同时还集成了多种视觉线索,包括形状、颜色和上下文等.利用该模型进行遥感影像中的目标检测与定位的结果表明,该模型能有效的检测出目标并给出目标所在的位置.     

基于改进贝叶斯方法的轨迹预测算法研究

针对传统轨迹预测方法在历史轨迹数目有限时,预测准确度较低的问题,提出一种改进的贝叶斯推理(MBI)方法,MBI构建了马尔可夫模型来量化相邻位置的相关性,并通过对历史轨迹进行分解来获得更准确的马尔可夫模型,最后得到改进的贝叶斯推理公式。实验结果表明,MBI方法比现有方法的预测速度快2到3倍,并且有较高的准确度和稳定性。MBI方法充分利用现有轨迹信息,不仅提高了查询效率,还保证了较高的预测精度。     

基于局部变分贝叶斯推断的分布式交互式多模型估计

针对目前部分多模型算法预先设定运动模型转移概率矩阵对状态估计精度的不利影响,本文提出了一种基于局部变分贝叶斯推断的分布式交互式多模型估计算法.不同于传统交互式多模型估计中运动模型转移概率矩阵为先验已知的假设条件,在分布融合估计框架下,首先基于最小化Kullback-Leibler散度准则的递归优化策略实现对运动模型转移概率矩阵的预测与更新;在此基础上,结合变分贝叶斯推断实现对当前时刻目标状态与模型概率的联合估计;最后依据协方差交叉融合策略完成对局部状态估计融合.仿真结果表明:新算法通过对运动模型转移概率矩阵以及模型概率自适应在线估计,有效提升了机动目标的状态估计精度.     

基于高斯混合聚类的切换系统的辨识

针对具有未知切换规则与未知子系统数量的切换系统的辨识问题,提出一种两阶段辨识方法,包括模式检测与参数辨识.在模式检测阶段,首先建立高斯混合模型表示采样数据的分布,并通过轮盘法选择合适的初始模型参数.其次,计算采样数据属于每个子系统的后验概率,通过极大似然估计算法迭代更新模型参数,使高斯混合模型最大化地拟合采样数据的分布.在此基础上,通过贝叶斯信息准则确定子系统的数量,并根据最大后验概率准则估计切换规则.在参数辨识阶段,通过递推增广最小二乘法估计每个子系统的参数向量.最后,通过仿真结果验证了所提方法的有效性.     

Discrete data clustering using finite mixture models

Finite mixture models have been applied for different computer vision, image processing and pattern recognition tasks. The majority of the work done concerning finite mixture models has focused on mixtures for continuous data. However, many applications involve and generate discrete data for which discrete mixtures are better suited. In this paper, we investigate the problem of discrete data modeling using finite mixture models. We propose a novel, well motivated mixture that we call the multinomial generalized Dirichlet mixture. The novel model is compared with other discrete mixtures. We designed experiments involving spatial color image databases modeling and summarization, and text classification to show the robustness, flexibility and merits of our approach.     

基于改进高斯混合模型的实时运动目标检测与跟踪*

为提高运动目标检测与跟踪的可靠性,提出了一种基于改进高斯混合模型的实时运动目标检测与跟踪算法。该算法建立可自动调节分布数目的高斯混合背景模型,通过背景减除获取前景图像;利用目标相邻帧的连续性分割运动目标;在此基础上将传统的颜色直方图模型进行改进,提高目标颜色分布的可信度,进而根据目标的位置、大小和颜色构造运动目标全局匹配相似度函数,实时完成运动目标检测与跟踪。利用大量的监控视频数据进行验证,结果表明,与传统的检测跟踪算法相比,该算法减少了计算量,提高了复杂背景情况下运动目标检测与跟踪的可靠性。     

基于GMM和贝叶斯推理的多模态过程运行状态评价

为使综合经济效益最大化,生产过程应保持在最优运行状态等级.针对多模态过程运行状态等级优劣判断问题,提出一种运行状态等级评价方法.该方法对同一运行状态等级的多模态数据建立一个高斯混合模型(Gaussian mixture model,GMM),确保特征提取的准确性,避免模态划分问题.至于在线评价策略,本文采用贝叶斯推理,确定当前运行状态属于各等级的后验概率.并引入滑动窗口,判定当前运行状态等级,有效解决多模态过程运行状态在线评价问题.针对"非优"运行状态,本文提出一种基于变量偏导数的贡献计算方法,对导致过程运行状态等级"非优"的原因变量进行追溯.最后,通过田纳西–伊斯曼(Tennessee–Eastman,TE)过程验证所提方法的有效性.     

A cluster analysis of vote transitions

To help settle the debate triggered the day after any election around the origin and destination of the vote of winners and losers, a Bayesian analysis of the results in a pair of consecutive elections is proposed. It is based on a model that simultaneously carries out a cluster analysis of the areas in which the results are broken into and links the results in the two elections of areas in a given cluster through a vote switch matrix. The number of clusters is chosen both through predictive checks as well as by testing whether the residuals are spatially correlated or not. The analysis is tried on the results in Barcelona of a pair of consecutive elections held just four months apart, in 2003 for the Catalan parliament and in 2004 for the Spanish parliament. The proposed approach, which reconstructs individual behavior from aggregated data, can be exported to be a solution for any ecological inference problem where one cannot assume that all the areas are exchangeable the way typically assumed by other ecological inference methods.     

基于分类样本和贝叶斯动态预测的异常入侵检测

在大规模网络环境中,入侵检测系统得到的警报数据具有一定的规律。据此提出了一种基于警报事件强度的异常检测方法,采用分类样本空间和贝叶斯动态预测方法,解决了警报数据的时间效应问题。实验数据分析表明,该方法对于大规模入侵行为具有较好的检测效果。     

Representing and processing lineages over uncertain data based on the Bayesian network

Processing lineages (also called provenances) over uncertain data consists in tracing the origin of uncertainty based on the process of data production and evolution. In this paper, we focus on the representation and processing of lineages over uncertain data, where we adopt Bayesian network (BN), one of the popular and important probabilistic graphical models (PGMs), as the framework of uncertainty representation and inferences. Starting from the lineage expressed as Boolean formulae for SPJ (Selection–Projection–Join) queries over uncertain data, we propose a method to transform the lineage expression into directed acyclic graphs (DAGs) equivalently. Specifically, we discuss the corresponding probabilistic semantics and properties to guarantee that the graphical model can support effective probabilistic inferences in lineage processing theoretically. Then, we propose the function-based method to compute the conditional probability table (CPT) for each node in the DAG. The BN for representing lineage expressions over uncertain data, called lineage BN and abbreviated as LBN, can be constructed while generally suitable for both safe and unsafe query plans. Therefore, we give the variable-elimination-based algorithm for LBN's exact inferences to obtain the probabilities of query results, called LBN-based query processing. Then, we focus on obtaining the probabilities of inputs or intermediate tuples conditioned on query results, called LBN-based inference query processing, and give the Gibbs-sampling-based algorithm for LBN's approximate inferences. Experimental results show the efficiency and effectiveness of our methods.     

基于贝叶斯分类算法的客户流失分析模型研究

客户关系管理以客户为中心,通过再造企业组织体系和优化业务流程,展开系统的客户研究,最大程度地改善、提高了整个客户关系生命周期的绩效,从而提高客户的满意度和忠诚度,提高运营效率和利润收益。该文研究和探讨了客户关系管理系统开发的技术环节及实现过程,并对基于贝叶斯分类算法的客户流失分析模型的建立进行了分析。     

基于贝叶斯推理的HCM延误模型修正

针对以1个周期时长为分析单位、使用HCM2000延误模型推导信号控制交叉口延误的问题,提出推导模型中参数修正的方法,用t检验验证参数提取的精度。对延误提取模型中的饱和度、启动损失时间及交叉口几何修正系数等参数进行分析,采用贝叶斯定理和马尔科夫链蒙特卡罗模拟方法对参数进行修正。结果证明该方法可以提高按照周期提取延误参数的精度。     

A Sequential Monte Carlo Method for Bayesian Analysis of Massive Datasets

Markov chain Monte Carlo (MCMC) techniques revolutionized statistical practice in the 1990s by providing an essential toolkit for making the rigor and flexibility of Bayesian analysis computationally practical. At the same time the increasing prevalence of massive datasets and the expansion of the field of data mining has created the need for statistically sound methods that scale to these large problems. Except for the most trivial examples, current MCMC methods require a complete scan of the dataset for each iteration eliminating their candidacy as feasible data mining techniques.In this article we present a method for making Bayesian analysis of massive datasets computationally feasible. The algorithm simulates from a posterior distribution that conditions on a smaller, more manageable portion of the dataset. The remainder of the dataset may be incorporated by reweighting the initial draws using importance sampling. Computation of the importance weights requires a single scan of the remaining observations. While importance sampling increases efficiency in data access, it comes at the expense of estimation efficiency. A simple modification, based on the rejuvenation step used in particle filters for dynamic systems models, sidesteps the loss of efficiency with only a slight increase in the number of data accesses.To show proof-of-concept, we demonstrate the method on two examples. The first is a mixture of transition models that has been used to model web traffic and robotics. For this example we show that estimation efficiency is not affected while offering a 99% reduction in data accesses. The second example applies the method to Bayesian logistic regression and yields a 98% reduction in data accesses.