Upper bound for variational free energy of Bayesian networks

In recent years, variational Bayesian learning has been used as an approximation of Bayesian learning. In spite of the computational tractability and good generalization in many applications, its statistical properties have yet to be clarified. In this paper, we focus on variational Bayesian learning of Bayesian networks which are widely used in information processing and uncertain artificial intelligence. We derive upper bounds for asymptotic variational free energy or stochastic complexities of bipartite Bayesian networks with discrete hidden variables. Our result theoretically supports the effectiveness of variational Bayesian learning as an approximation of Bayesian learning.     

二值probit回归模型的坍缩变分贝叶斯推断算法

给出了二值probit回归模型的坍缩变分贝叶斯推断算法.此算法比变分贝叶斯推断算法能更逼近对数边缘似然,得到更精确的模型参数后验期望值.如果两个算法得到的分类错误一致,则该算法的迭代次数较变分法明显减少.仿真实验结果验证了所提出算法的有效性.     

Robust 1-bit compressive sensing via variational Bayesian algorithm

In a compressive sensing (CS) framework, a sparse signal can be stably reconstructed at a reduced sampling rate. Quantization and noise corruption are inevitable in practical applications. Recent studies have shown that using only the sign information of measurements can achieve accurate signal reconstruction in a CS framework. We consider the problem of reconstructing a sparse signal from 1-bit quantized, Gaussian noise corrupted measurements. In this paper, we present a variational Bayesian inference based 1-bit compressive sensing algorithm, which essentially models the effect of quantization as well as the Gaussian noise. A variational message passing method is adopted to achieve the inference. Through numerical experiments, we demonstrate that our algorithm outperforms state-of-the-art 1-bit compressive sensing algorithms in the presence of Gaussian noise corruption.     

A note on mean-field variational approximations in Bayesian probit models

We correct some conclusions presented by Consonni and Marin (2007) on the performance of mean-field variational approximations to Bayesian inferences in the case of a simple probit model. We show that some of their presentations are misleading and thus their results do not fairly present the performance of such approximations in terms of point estimation under the specified model.     

Bayesian estimation of Dirichlet mixture model with variational inference

In statistical modeling, parameter estimation is an essential and challengeable task. Estimation of the parameters in the Dirichlet mixture model (DMM) is analytically intractable, due to the integral expressions of the gamma function and its corresponding derivatives. We introduce a Bayesian estimation strategy to estimate the posterior distribution of the parameters in DMM. By assuming the gamma distribution as the prior to each parameter, we approximate both the prior and the posterior distribution of the parameters with a product of several mutually independent gamma distributions. The extended factorized approximation method is applied to introduce a single lower-bound to the variational objective function and an analytically tractable estimation solution is derived. Moreover, there is only one function that is maximized during iterations and, therefore, the convergence of the proposed algorithm is theoretically guaranteed. With synthesized data, the proposed method shows the advantages over the EM-based method and the previously proposed Bayesian estimation method. With two important multimedia signal processing applications, the good performance of the proposed Bayesian estimation method is demonstrated.     

Variational Bayesian noise estimation of point sets

Scanning devices acquire geometric information from the surface of an object in the form of a 3D point set. Such point sets, as any data obtained by means of physical measurement, contain some noise. To create an accurate model of the scanned object, this noise should be resolved before or during the process of surface reconstruction. In this paper, we develop a statistical technique to estimate the noise in a scanned point set. The noise is represented as normal distributions with zero mean and their variances determine the amount of the noise. These distributions are estimated with a variational Bayesian method, which is known to provide more robust estimations than point estimate methods, such as maximum likelihood and maximum a posteriori. Validation experiments and further tests with real scan data show that the proposed technique can accurately estimate the noise in a 3D point set.     

Robust monitoring and fault reconstruction based on variational inference component analysis

Probabilistic models such as probabilistic principal component analysis (PPCA) have recently caught much attention in the process monitoring area. An important issue of the PPCA method is how to determine the dimensionality of the latent variable space. In the present paper, one of the most popular Bayesian type chemometric methods, Bayesian PCA (BPCA) is introduced for process monitoring purpose, which is based on the recent developed variational inference algorithm. In this monitoring framework, the effectiveness of each extracted latent variable can be well reflected by a hyperparameter, upon which the dimensionality of the latent variable space can be automatically determined. Meanwhile, for practical consideration, the developed BPCA-based monitoring method is robust to missing data and can also give satisfactory performance under limited data samples. Another contribution of this paper is due to the proposal of a new fault reconstruction method under the BPCA model structure. Two case studies are provided to evaluate the performance of the proposed method.     

面向污水处理的动态变分贝叶斯混合因子故障诊断

在污水生化处理过程中,存在着多变量耦合、强非线性、参数时变、大滞后等特点,面对这些特点,传感器故障频发,从而导致生化过程无法得到有效优化和诊断.为此,本文在结合动态数据特性的基础上提出了一种基于变分贝叶斯混合因子的动态故障诊断方法,同时,利用混合因子的在线调整实现了诊断模型的半自适应化.该方法能够捕捉到污水处理过程的强非线性和动态性,从而可有效降低故障诊断的误报率和漏报率.通过在国际水协会的BSM1模型上的模拟研究,充分表明所提出的策略可以显著提高故障诊断能力,精确地检测传感器的突变和漂移故障,甚至定位故障所发生的根本原因.     

Bayesian wavelet denoising and evolutionary calibration

Wavelet shrinkage estimation has become an attractive and efficient method for signal denoising and compression. Despite the ample variety of methods which have been used in the wavelet denoising context, it has proven elusive to construct threshold estimators with good adaptive properties. Recently, empirical Bayes selection criteria have been proposed to derive adaptive shrinkage estimators. We consider the application of empirical Bayes variable selection criteria to each level of the wavelet transform to obtain adaptive threshold estimates. A set of level-dependent hyperparameters has to be estimated to derive nonlinear data-dependent thresholding rules. We propose the use of an evolutionary algorithm to calibrate the multilevel parameters, in order to automate parameter selection and enhance adaptivity of the threshold estimators. Comparative simulations on a set of standard model functions show good performance. Applications to data drawn from various fields of application are used to explore the practical performance of the proposed approach.     

一般贝叶斯网络分类器及其学习算法

贝叶斯网络（BN）应用于分类应用时对目标变量预测有直接贡献的局部模型称作一般贝叶斯网络分类器（GBNC）。推导GBNC的传统途径是先学习完整的BN,而现有推导BN结构的算法限制了应用规模。为了避免学习全局BN,提出仅执行局部搜索的结构学习算法IPC-GBNC,它以目标变量节点为中心执行广度优先搜索,且将搜索深度控制在不超过2层。理论上可证明算法IPC-GBNC是正确的,而基于仿真和真实数据的实验进一步验证了其学习效果和效率的优势：（1）可输出和执行全局搜索的PC算法相同甚至更高质量的结构;（2）较全局搜索消耗少得多的计算量;（3）同时实现了降维（类似决策树学习算法）。相比于绝大多数经典分类器,GBNC的分类性能相当,但兼具直观、紧凑表达和强大推理的能力（且支持不完整观测值）。     

A variational Bayesian approach for the robust analysis of the cortical silent period from EMG recordings of brain stroke patients

Transcranial magnetic stimulation (TMS) is a powerful tool for the calculation of parameters related to the intracortical excitability and inhibition of the motor cortex. The cortical silent period (CSP) is one such parameter that corresponds to the suppression of muscle activity for a short period after a muscle response to TMS. The duration of the CSP is known to be correlated with the prognosis of brain stroke patients' motor ability. Current methods for the estimation of the CSP duration are very sensitive to the presence of noise. A variational Bayesian formulation of a manifold-constrained hidden Markov model is applied in this paper to the segmentation of a set of multivariate time series (MTS) of electromyographic recordings corresponding to stroke patients and control subjects. A novel index of variability associated to this model is defined and applied to the detection of the silent period interval of the signal and to the estimation of its duration. This model and its associated index are shown to behave robustly in the presence of noise and provide more reliable estimations than the current standard in clinical practice.     

A variational Bayesian strategy for solving the DOA estimation problem in sparse array

This paper reformulates the problem of direction-of-arrival (DOA) estimation for sparse array from a variational Bayesian perspective. In this context, we propose a hierarchical prior for the signal coefficients that amounts marginally to a sparsity-inducing penalty in maximum a posterior (MAP) estimation. Further, the specific hierarchy gives rise to a variational inference technique which operates in latent variable space iteratively. Our hierarchical formulation of the prior allow users to model the sparsity of the unknown signal with a high degree, and the corresponding Bayesian algorithm leads to sparse estimators reflecting posterior information beyond the mode. We provide experimental results with synthetic signals and compare with state-of-the-art DOA estimation algorithm, in order to demonstrate the superior performance of the proposed approach.     

A Bayesian approach to model-based clustering for binary panel probit models

Considering latent heterogeneity is of special importance in nonlinear models in order to gauge correctly the effect of explanatory variables on the dependent variable. A stratified model-based clustering approach is adapted for modeling latent heterogeneity in binary panel probit models. Within a Bayesian framework an estimation algorithm dealing with the inherent label switching problem is provided. Determination of the number of clusters is based on the marginal likelihood and a cross-validation approach. A simulation study is conducted to assess the ability of both approaches to determine on the correct number of clusters indicating high accuracy for the marginal likelihood criterion, with the cross-validation approach performing similarly well in most circumstances. Different concepts of marginal effects incorporating latent heterogeneity at different degrees arise within the considered model setup and are directly at hand within Bayesian estimation via MCMC methodology. An empirical illustration of the methodology developed indicates that consideration of latent heterogeneity via latent clusters provides the preferred model specification over a pooled and a random coefficient specification.     

A variational Bayesian methodology for hidden Markov models utilizing Student's-t mixtures

The Student's-t hidden Markov model (SHMM) has been recently proposed as a robust to outliers form of conventional continuous density hidden Markov models, trained by means of the expectation-maximization algorithm. In this paper, we derive a tractable variational Bayesian inference algorithm for this model. Our innovative approach provides an efficient and more robust alternative to EM-based methods, tackling their singularity and overfitting proneness, while allowing for the automatic determination of the optimal model size without cross-validation. We highlight the superiority of the proposed model over the competition using synthetic and real data. We also demonstrate the merits of our methodology in applications from diverse research fields, such as human computer interaction, robotics and semantic audio analysis.     

Combining feature spaces for classification

In this paper we offer a variational Bayes approximation to the multinomial probit model for basis expansion and kernel combination. Our model is well-founded within a hierarchical Bayesian framework and is able to instructively combine available sources of information for multinomial classification. The proposed framework enables informative integration of possibly heterogeneous sources in a multitude of ways, from the simple summation of feature expansions to weighted product of kernels, and it is shown to match and in certain cases outperform the well-known ensemble learning approaches of combining individual classifiers. At the same time the approximation reduces considerably the CPU time and resources required with respect to both the ensemble learning methods and the full Markov chain Monte Carlo, Metropolis-Hastings within Gibbs solution of our model. We present our proposed framework together with extensive experimental studies on synthetic and benchmark datasets and also for the first time report a comparison between summation and product of individual kernels as possible different methods for constructing the composite kernel matrix.     

A novel Bayesian deep learning methodology for enhanced foetal cardiac signal mining

Cardiac defects are amongst the most common birth defects. Cardiac diagnosis is indispensably imperative in the foetal stage as it might help provide an opportunity to plan and manage the baby during Antepartum and Intrapartum stages, when the baby is born. It is from the Antepartum stage where the foetal electrocardiogram (fECG) signal can actually be detected. At present, monitoring the foetus is completely focused on the heart rate. Currently fECG analysis is used in the clinical domain to analyse heart rate and the allied variations. Analysis using the morphology of the fECG is generally not undertaken for cardiac-anomaly populations. The ultimate reason for this scenario is due to unavailability in technology to yield trustworthy fECG measurements with desired quality required by Physicians. A novel hybrid methodology called BDL (Bayesian Deep Learning) methodology is proposed. The BDL includes a Bayesian filter and a deep learning (DL) Artificial Intelligent neural network for maternal electrocardiogram (mECG) elimination and non-linear artefacts removal to yield high quality non-invasive fECG signal. The outcomes of the research by the proposed BDL system proved valuable and provided high quality fECG signal for efficient foetal diagnosis.