Analysis of single Gaussian approximation of Gaussian mixtures in Bayesian filtering applied to mixed multiple-model estimation

This paper examines the effect of the moment-matched single Gaussian approximation, which is made in various multiple-model filtering applications to approximate a Gaussian mixture, on the Bayesian filter performance. The estimation error caused by the approximation is analysed for both the prediction and the measurement updates of a Bayesian filter. An approximate formula is found for the covariance of the error caused by the approximation for a general Gaussian mixture with arbitrary components. The calculated error covariance is used for obtaining a mixed multiple-model estimation algorithm which has a performance near that of GPB2 with less computations.     

基于DEnKF的背景误差协方差局地化和协方差膨胀研究

尽管DEnKF同化不会引入观测采样误差,但小集合仍会造成背景误差协方差矩阵存在伪相关,出现滤波发散。为了减少小集合对数据同化结果的影响,结合Lorenz96模型和DEnKF同化方案分析了协方差局地化和协方差膨胀方法对背景误差协方差矩阵、增益矩阵及同化结果的影响。实验表明:协方差局地化方法能消除背景误差协方差矩阵和增益矩阵中的伪相关,增大背景误差协方差矩阵的秩,有助于滤波算法收敛到真实解;而协方差膨胀方法不能消除背景误差协方差矩阵和增益矩阵中的伪相关,只能改善在每个同化周期内背景误差协方差系统性被低估的现象;同化过程中采用合适的局地化半径和方差膨胀因子能够较好地改善同化结果的精度。     

Information-theoretic Approaches Based on Sequential Monte Carlo to Collaborative Distributed Sensors for Mobile Robot Localization

We consider the Sequential Monte Carlo (SMC) method for Bayesian inference applied to the problem of information-theoretic distributed sensor collaboration in complex environments. The robot kinematics and sensor observation under consideration are described by nonlinear models. The exact solution to this problem is prohibitively complex due to the nonlinear nature of the system. The SMC method is, therefore, employed to track the probabilistic kinematics of the robot and to make the corresponding Bayesian estimates and predictions. To meet the specific requirements inherent in distributed sensors, such as low-communication consumption and collaborative information processing, we propose a novel SMC solution that makes use of the particle filter technique for data fusion, and the density tree representation of the a posterior distribution for information exchange between sensor nodes. Meanwhile, an efficient numerical method is proposed for approximating the information utility in sensor selection. A further experiment, obtained with a real robot in an indoor environment, illustrates that under the SMC framework, the optimal sensor selection and collaboration can be implemented naturally, and significant improvement in localization accuracy is achieved when compared to conventional methods using all sensors.     

Statistical Estimation and Classification on Commutative Covariance Structures

Statistical inference is investigated under the following constraints on the covariance structure for the observation vector: covariance matrices belong to some commutative matrix algebra. Commutative approximation of arbitrary covariance structures and statistical estimation of the parameters of a given commutative structure are studied. The results are applied to statistical classification of Gaussian vectors having commutative covariance structure.     

基于完全联系的条件随机场的图像标注

传统的图像标注模型通常存在两个问题：只能够对短距离的像素上下文信息进行建模和复杂的模型推理过程。为了提高图像标注的精度、简化图像标注的模型推理过程,采用完全联系的条件随机场模型进行图像标注,提出利用基于高斯kd树的平均场估计方法实现该模型的高效推理。为了更好地验证算法的有效性,实验的图片数据库不仅包含标准的图片库--剑桥大学微软研究图片库（MSRC-9）,还包含作者制作的机械零件图片库（MyDataset_1）和办公桌图片库（MyDataset_2）。新算法在三个图片库上的平均标注精度分别可以达到77.96%、97.15%和95.35%,每幅图的平均运行时间为2s。实验结果表明,基于完全联系的条件随机场的图像标注能够更充分地考虑不同的像素上下文信息来提高标注精度,而基于高斯kd树的模型推理能够提高模型推理的效率。     

Optimizing the stretch of independent tasks on a cluster: From sequential tasks to moldable tasks

This paper addresses the problem of scheduling non-preemptive moldable tasks to minimize the stretch of the tasks in an online non-clairvoyant setting. To the best of the authors’ knowledge, this problem has never been studied before. To tackle this problem, first the sequential subproblem is studied through the lens of the approximation theory. An algorithm, called DASEDF, is proposed and, through simulations, it is shown to outperform the first-come, first-served scheme. Furthermore, it is observed that machine availability is the key to getting good stretch values. Then, the moldable task scheduling problem is considered, and, by leveraging the results from the sequential case, another algorithm, DBOS, is proposed to optimize the stretch while scheduling moldable tasks. This work is motivated by a task scheduling problem in the context of parallel short sequence mapping which has important applications in biology and genetics. The proposed DBOS algorithm is evaluated both on synthetic data sets that represent short sequence mapping requests and on data sets generated using log files of real production clusters. The results show that the DBOS algorithm significantly outperforms the two state-of-the-art task scheduling algorithms on stretch optimization.     

二值probit回归模型的坍缩变分贝叶斯推断算法

给出了二值probit回归模型的坍缩变分贝叶斯推断算法.此算法比变分贝叶斯推断算法能更逼近对数边缘似然,得到更精确的模型参数后验期望值.如果两个算法得到的分类错误一致,则该算法的迭代次数较变分法明显减少.仿真实验结果验证了所提出算法的有效性.     

Variational Bayesian sparse additive matrix factorization

Principal component analysis (PCA) approximates a data matrix with a low-rank one by imposing sparsity on its singular values. Its robust variant can cope with spiky noise by introducing an element-wise sparse term. In this paper, we extend such sparse matrix learning methods, and propose a novel framework called sparse additive matrix factorization (SAMF). SAMF systematically induces various types of sparsity by a Bayesian regularization effect, called model-induced regularization. Although group LASSO also allows us to design arbitrary types of sparsity on a matrix, SAMF, which is based on the Bayesian framework, provides inference without any requirement for manual parameter tuning. We propose an efficient iterative algorithm called the mean update (MU) for the variational Bayesian approximation to SAMF, which gives the global optimal solution for a large subset of parameters in each step. We demonstrate the usefulness of our method on benchmark datasets and a foreground/background video separation problem.     

Sequential fusion estimation for multisensor multirate systems with measurement outliers

This paper investigates the suboptimal sequential fusion estimation problem for multisensor multirate networked systems with colored measurement noises under the interference of measurement outliers. The saturation function is used to constrain the innovation polluted by measurement outliers. Due to diverse physical restrictions, the sampling period of the sensor is assumed to be different from the update period of the system state, thereby better reflecting the engineering practice. The lifting technique is used to convert the multirate sampling system into a single-rate form. By solving the matrix difference equation, an upper bound of the filtering error covariance is obtained, and the filter gain is then derived, which can minimize the upper bound of the error covariance. Finally, a simulation example is given to demonstrate the effectiveness of the proposed sequential fusion method for multirate sampling systems under outlier interference.     

Kalman filtering with intermittent observations

Motivated by navigation and tracking applications within sensor networks, we consider the problem of performing Kalman filtering with intermittent observations. When data travel along unreliable communication channels in a large, wireless, multihop sensor network, the effect of communication delays and loss of information in the control loop cannot be neglected. We address this problem starting from the discrete Kalman filtering formulation, and modeling the arrival of the observation as a random process. We study the statistical convergence properties of the estimation error covariance, showing the existence of a critical value for the arrival rate of the observations, beyond which a transition to an unbounded state error covariance occurs. We also give upper and lower bounds on this expected state error covariance.     

基于变分贝叶斯逼近的前向仿射变换点集匹配方法研究

本文建立了两个点集线性匹配过程的贝叶斯模型框架,并利用变分贝叶斯逼近方法对模型点集到场景点集的仿射参数进行估计。该模型利用一个有向图对映射参数、隐藏变量、模型与场景点集的关系进行了描述,并基于有向图给出了各个参数和变量后验概率的迭代估计算法。而且该模型还利用了一个带有各向异性协方差矩阵的高斯模型对场景点集的离群点进行了估计和推理。实验结果表明该模型在鲁棒性和匹配精度方面均获得了良好的效果。     

Sensor planning for mobile robot localization using Bayesian network inference

We propose a new method of sensor planning for mobile robot localization using Bayesian network inference. Since we can model causal relations between situations of the robot's behavior and sensing events as nodes of a Bayesian network, we can use the inference of the network for dealing with uncertainty in sensor planning and thus derive appropriate sensing actions. In this system we employ a multi-layered-behavior architecture for navigation and localization. This architecture effectively combines mapping of local sensor information and the inference via a Bayesian network for sensor planning. The mobile robot recognizes the local sensor patterns for localization and navigation using a learned regression function. Since the environment may change during the navigation and the sensor capability has limitations in the real world, the mobile robot actively gathers sensor information to construct and reconstruct a Bayesian network, and then derives an appropriate sensing action which maximizes a utility function based on inference of the reconstructed network. The utility function takes into account belief of the localization and the sensing cost. We have conducted some simulation and real robot experiments to validate the sensor planning system.     

An architecture for generic extensions

We examine what is necessary to allow generic libraries to be used naturally in a multi-language, potentially distributed environment. Language-neutral library interfaces usually do not support the full range of programming idioms that are available when a library is used natively. We investigate how to structure the language bindings of the neutral interface to achieve a better expressibility and code re-use. We furthermore address how language-neutral interfaces can be extended with import bindings to recover the desired programming idioms. We also address the question of how these extensions can be organized to minimize the performance overhead that arises from using objects in manners not anticipated by the original library designers. Our approach is to treat a library as a software component and to view the problem as one of component extension. We use C++ as an example of a mature language, with libraries using a variety of patterns, and use the Standard Template Library as an example of a complex library for which efficiency is important. By viewing the library extension problem as one of component organization, we enhance software composibility, hierarchy maintenance and architecture independence.     

磁偶极子跟踪的渐进贝叶斯滤波方法

提出一种新的非线性滤波器应用于磁偶极子目标跟踪问题.建立了跟踪问题的状态空间模型, 在此基础上, 从高斯矩近似误差的角度分析了现有卡尔曼滤波更新在磁偶极子跟踪中的问题.对此, 将贝叶斯更新过程等效为求解连续时间上的渐进贝叶斯问题, 在线性高斯条件下推导了其解析解, 表明渐进贝叶斯更新可等效为定常系统的Kalman-Bucy滤波器; 进一步采用一阶Taylor展开得到非线性近似解表达式, 导出一种渐进贝叶斯滤波器, 从理论上分析了与现有方法的异同.仿真与实测磁目标跟踪试验结果表明, 渐进贝叶斯滤波器具有良好的精度和收敛性, 能够有效抑制磁目标跟踪中由于大初始误差导致的性能下降和滤波发散, 且计算效率与扩展卡尔曼滤波器相当, 适于实际应用.     

Bayesian zero-order TSK fuzzy system modeling

Different from the existing TSK fuzzy system modeling methods, a novel zero-order TSK fuzzy modeling method called Bayesian zero-order TSK fuzzy system (B-ZTSK-FS) is proposed from the perspective of Bayesian inference in this paper. The proposed method B-ZTSK-FS constructs zero-order TSK fuzzy system by using the maximum a posteriori (MAP) framework to maximize the corresponding posteriori probability. First, a joint likelihood model about zero-order TSK fuzzy system is defined to derive a new objective function which can assure that both antecedents and consequents of fuzzy rules rather than only their antecedents of the most existing TSK fuzzy systems become interpretable. The defined likelihood model is composed of three aspects: clustering on the training set for antecedents of fuzzy rules, the least squares (LS) error for consequent parameters of fuzzy rules, and a Dirichlet prior distribution for fuzzy cluster memberships which is considered to not only automatically match the “sum-to-one” constraints on fuzzy cluster memberships, but also make the proposed method B-ZTSK-FS scalable for large-scale datasets by appropriately setting the Dirichlet index. This likelihood model indeed indicates that antecedent and consequent parameters of fuzzy rules can be linguistically interpreted and simultaneously optimized by the proposed method B-ZTSK-FS which is based on the MAP framework with the iterative sampling algorithm, which in fact implies that fuzziness and probability can co-jointly work for TSK fuzzy system modeling in a collaborative rather than repulsive way. Finally, experimental results on 28 synthetic and real-world datasets are reported to demonstrate the effectiveness of the proposed method B-ZTSK-FS in the sense of approximation accuracy, interpretability and scalability.     

Improved inference on a scalar fixed effect of interest in nonlinear mixed-effects models

Likelihood-based inference on a scalar fixed effect of interest in nonlinear mixed-effects models usually relies on first-order approximations. If the sample size is small, tests and confidence intervals derived from first-order solutions can be inaccurate. An improved test statistic based on a modification of the signed likelihood ratio statistic is presented which was recently suggested by Skovgaard [1996. An explicit large-deviation approximation to one-parameter tests. Bernoulli 2, 145-165]. The finite sample behaviour of this statistic is investigated through a set of simulation studies. The results show that its finite-sample null distribution is better approximated by the standard normal than it is for its first-order counterpart. The R code used to run the simulations is freely available.     

A Comparison between the Tikhonov and the Bayesian Approaches to Calculate Regularisation Matrices

Regularisation is a well-known technique for working with ill-posed and ill-conditioned problems that have been explored in a variety of different areas, including Bayesian inference, functional analysis, optimisation, numerical analysis and connectionist systems. In this paper we present the equivalence between the Bayesian approach to the regularisation theory and the Tikhonov regularisation into the function approximation theory framework, when radial basis functions networks are employed. This equivalence can be used to avoid expensive calculations when regularisation techniques are employed.     

Quantile regression neural networks: Implementation in R and application to precipitation downscaling

The qrnn package for R implements the quantile regression neural network, which is an artificial neural network extension of linear quantile regression. The model formulation follows from previous work on the estimation of censored regression quantiles. The result is a nonparametric, nonlinear model suitable for making probabilistic predictions of mixed discrete-continuous variables like precipitation amounts, wind speeds, or pollutant concentrations, as well as continuous variables. A differentiable approximation to the quantile regression error function is adopted so that gradient-based optimization algorithms can be used to estimate model parameters. Weight penalty and bootstrap aggregation methods are used to avoid overfitting. For convenience, functions for quantile-based probability density, cumulative distribution, and inverse cumulative distribution functions are also provided. Package functions are demonstrated on a simple precipitation downscaling task.     

Numerical object oriented quantum field theory calculations

The qft++ package is a library of C++ classes that facilitate numerical (not algebraic) quantum field theory calculations. Mathematical objects such as matrices, tensors, Dirac spinors, polarization and orbital angular momentum tensors, etc. are represented as C++ objects in qft++. The package permits construction of code which closely resembles quantum field theory expressions, allowing for quick and reliable calculations.     

Bayesian approach to neural-network modeling with input uncertainty

It is generally assumed when using Bayesian inference methods for neural networks that the input data contains no noise or corruption. For real-world (errors in variable) problems this is clearly an unsafe assumption. This paper presents a Bayesian neural-network framework which allows for input noise provided that some model of the noise process exists. In the limit where the noise process is small and symmetric it is shown, using the Laplace approximation, that this method gives an additional term to the usual Bayesian error bar which depends on the variance of the input noise process. Further, by treating the true (noiseless) input as a hidden variable and sampling this jointly with the network weights using a Markov chain Monte Carlo method, it is demonstrated that it is possible to infer the regression over the noiseless input.