A decision fusion method using an algorithm for fusion of correlated probabilities

This paper proposes a new decision fusion method accounting for conditional dependence (correlation) between land-cover classifications from multi-sensor data. The dependence structure between different classification results is calculated and used as weighting parameters for the subsequent fusion scheme. An algorithm for fusion of correlated probabilities (FCP) is adopted to fuse the prior probability, conditional probability, and obtained weighting parameters to generate a posterior probability for each class. A maximum posterior probability rule is then used to combine the posterior probabilities generated for each class to produce the final fusion result. The proposed FCP-based decision fusion method is assessed in land-cover classification over two study areas. The experimental results demonstrate that the proposed decision fusion method outperformed the existing decision fusion methods that do not take into account the correlation or dependence. The proposed decision fusion method can also be applied to other applications with different sensor data.     

Tree Induction for Probability-Based Ranking

Tree induction is one of the most effective and widely used methods for building classification models. However, many applications require cases to be ranked by the probability of class membership. Probability estimation trees (PETs) have the same attractive features as classification trees (e.g., comprehensibility, accuracy and efficiency in high dimensions and on large data sets). Unfortunately, decision trees have been found to provide poor probability estimates. Several techniques have been proposed to build more accurate PETs, but, to our knowledge, there has not been a systematic experimental analysis of which techniques actually improve the probability-based rankings, and by how much. In this paper we first discuss why the decision-tree representation is not intrinsically inadequate for probability estimation. Inaccurate probabilities are partially the result of decision-tree induction algorithms that focus on maximizing classification accuracy and minimizing tree size (for example via reduced-error pruning). Larger trees can be better for probability estimation, even if the extra size is superfluous for accuracy maximization. We then present the results of a comprehensive set of experiments, testing some straightforward methods for improving probability-based rankings. We show that using a simple, common smoothing method—the Laplace correction—uniformly improves probability-based rankings. In addition, bagging substantially improves the rankings, and is even more effective for this purpose than for improving accuracy. We conclude that PETs, with these simple modifications, should be considered when rankings based on class-membership probability are required.     

Modular neural networks for MAP classification of time series andthe partition algorithm

We apply the partition algorithm to the problem of time-series classification. We assume that the source that generates the time series belongs to a finite set of candidate sources. Classification is based on the computation of posterior probabilities. Prediction error is used to adaptively update the posterior probability of each source. The algorithm is implemented by a hierarchical, modular, recurrent network. The bottom (partition) level of the network consists of neural modules, each one trained to predict the output of one candidate source. The top (decision) level consists of a decision module, which computes posterior probabilities and classifies the time series to the source of maximum posterior probability. The classifier network is formed from the composition of the partition and decision levels. This method applies to deterministic as well as probabilistic time series. Source switching can also be accommodated. We give some examples of application to problems of signal detection, phoneme, and enzyme classification. In conclusion, the algorithm presented here gives a systematic method for the design of modular classification networks. The method can be extended by various choices of the partition and decision components.     

Cost-sensitive and modular land-cover classification based on posterior probability estimates

Many types of nonlinear classifiers have been proposed to automatically generate land-cover maps from satellite images. Some are based on the estimation of posterior class probabilities, whereas others estimate the decision boundary directly. In this paper, we propose a modular design able to focus the learning process on the decision boundary by using posterior probability estimates. To do so, we use a self-configuring architecture that incorporates specialized modules to deal with conflicting classes, and we apply a learning algorithm that focuses learning on the posterior probability regions that are critical for the performance of the decision problem stated by the user-defined misclassification costs. Moreover, we show that by filtering the posterior probability map, the impulsive noise, which is a common effect in automatic land-cover classification, can be significantly reduced. Experimental results show the effectiveness of the proposed solutions on real multi- and hyperspectral images, versus other typical approaches, that are not based on probability estimates, such as Support Vector Machines.     

A computer simulation of medical decision strategy performance

This study demonstrates a computer model which can be used to compare the effects of errors in probability and utility estimation on the performance of Bayesian and alternative medical decision strategies. The model task requires choosing one of three treatments for a patient with one of three diseases based on the patient's state with respect to five binary cues and estimates of the prior probabilities of disease, the conditional probabilities of the cues and the utilities of the treatments. A classic decision analytic strategy uses Bayes' formula to calculate posterior probabilities of disease and chooses treatments based on maximization of expected value. Alternative strategies use likelihood ratios to calculate disease scores for each patient state and choose the treatment with highest payoff for the disease with the highest score. Two strategies with different cutoffs for the ratios are compared with a random strategy and a classic decision analytic strategy. The simulation results show that the payoffs for all strategies except the random strategy decline with increasing estimation error. The decision analytic strategy has the highest mean payoff at all levels of error. The differences between this optical strategy and the alternatives, however, decrease as estimation error increases, and the frequency with which the strategies based on simple diagnostic scoring rules outperform the formal Bayesian strategy increases.     

Local estimation of posterior class probabilities to minimize classification errors

Decision theory shows that the optimal decision is a function of the posterior class probabilities. More specifically, in binary classification, the optimal decision is based on the comparison of the posterior probabilities with some threshold. Therefore, the most accurate estimates of the posterior probabilities are required near these decision thresholds. This paper discusses the design of objective functions that provide more accurate estimates of the probability values, taking into account the characteristics of each decision problem. We propose learning algorithms based on the stochastic gradient minimization of these loss functions. We show that the performance of the classifier is improved when these algorithms behave like sample selectors: samples near the decision boundary are the most relevant during learning.     

多预测子融合实时连续语音识别输出词正误判别

本文在采用堆栈译码词网重估输出作为识别最终输出的连续语音识别实时解码条件下,利用决策树方法将多个预测子融合,对识别输出词进行正确和错误的判别。本文首先构造了词后验概率、词长、相邻词的后验概率、词的声学和语言得分等共13 个预测子,然后利用决策树方法,通过选择不同的预测子组合方式和适当的决策树建树参数,筛选出预测子的最佳组合,建立优化的决策树进行输出词的正误判别。实验结果表明:利用局域词图计算的词后验概率与词长、相邻词的后验概率等几种实时预测子融合后,对识别输出词的正误判别能力得到提高,并且在实时性和分类效果两个方面优于n - best 输出的相应结果,相对于基线系统, 则分类错误率下降41. 4 %。实验结果也表明本文提出的相邻词的后验概率是相对重要的预测子。     

Cost-sensitive learning based on Bregman divergences

This paper analyzes the application of a particular class of Bregman divergences to design cost-sensitive classifiers for multiclass problems. We show that these divergence measures can be used to estimate posterior probabilities with maximal accuracy for the probability values that are close to the decision boundaries. Asymptotically, the proposed divergence measures provide classifiers minimizing the sum of decision costs in non-separable problems, and maximizing a margin in separable MAP problems.     

Decision making on the sole basis of statistical likelihood

This paper presents a new axiomatic decision theory for choice under uncertainty. Unlike Bayesian decision theory where uncertainty is represented by a probability function, in our theory, uncertainty is given in the form of a likelihood function extracted from statistical evidence. The likelihood principle in statistics stipulates that likelihood functions encode all relevant information obtainable from experimental data. In particular, we do not assume any knowledge of prior probabilities. Consequently, a Bayesian conversion of likelihoods to posterior probabilities is not possible in our setting. We make an assumption that defines the likelihood of a set of hypotheses as the maximum likelihood over the elements of the set. We justify an axiomatic system similar to that used by von Neumann and Morgenstern for choice under risk. Our main result is a representation theorem using the new concept of binary utility. We also discuss how ambiguity attitudes are handled. Applied to the statistical inference problem, our theory suggests a novel solution. The results in this paper could be useful for probabilistic model selection.     

基于音素评分模型的发音标准度评测研究

在计算机辅助语言学习系统中,后验概率是普通话水平测试(PSC)电子化系统衡量考生发音标准程度的重要指标,但后验概率与人工的主观评分存在着显著差别。该文提出了“音素评分模型”的思想,对后验概率进行变换。该文研究了线性和非线性的sigmoid音素评分模型,并发现线性音素评分模型有闭式全局最优解,非线性音素评分模型可用梯度下降法求解。在全国采集的498人的普通话考试现场数据集上的实验表明该策略能使系统评分性能有明显的提升 当后验概率在全音素概率空间中计算时,可使系统性能提升约42%;当后验概率在优化的概率空间中计算时,能使系统性能提升约23%～27%。     

A diagnostic method that uses causal knowledge and linear programming in the application of Bayes' formula

Bayes' formula has been applied extensively in computer-based medical diagnostic systems. One assumption that is often made in the application of the formula is that the findings in a case are conditionally independent. This assumption is often invalid and leads to inaccurate posterior probability assignments to the diagnostic hypotheses. This paper discusses a method for using causal knowledge to structure findings according to their probabilistic dependencies. An inference procedure is discussed which propagates probabilities within a network of causally related findings in order to calculate posterior probabilities of diagnostic hypotheses. A linear programming technique is described that bounds the values of the propagated probabilities subject to known probabilistic constraints.     

Pattern Classification With Class Probability Output Network

The output of a classifier is usually determined by the value of a discriminant function and a decision is made based on this output which does not necessarily represent the posterior probability for the soft decision of classification. In this context, it is desirable that the output of a classifier be calibrated in such a way to give the meaning of the posterior probability of class membership. This paper presents a new method of postprocessing for the probabilistic scaling of classifier's output. For this purpose, the output of a classifier is analyzed and the distribution of the output is described by the beta distribution parameters. For more accurate approximation of class output distribution, the beta distribution parameters as well as the kernel parameters describing the discriminant function are adjusted in such a way to improve the uniformity of beta cumulative distribution function (CDF) values for the given class output samples. As a result, the classifier with the proposed scaling method referred to as the class probability output network (CPON) can provide accurate posterior probabilities for the soft decision of classification. To show the effectiveness of the proposed method, the simulation for pattern classification using the support vector machine (SVM) classifiers is performed for the University of California at Irvine (UCI) data sets. The simulation results using the SVM classifiers with the proposed CPON demonstrated a statistically meaningful performance improvement over the SVM and SVM-related classifiers, and also other probabilistic scaling methods.     

Decoding design based on posterior probabilities in Ternary Error-Correcting Output Codes

Ternary Error-Correcting Output Codes (ECOC), which can unify most of the state-of-the-art decomposition frameworks such as one-versus-one, one-versus-all, sparse coding, dense coding, etc., is considered more flexible to model multiclass classification problems than Binary ECOC. Meanwhile, there are many corresponding decoding strategies that have been proposed for Ternary ECOC in earlier literatures. Note that there is few working by posterior probabilities, which can be considered as a Bayes decision rule and hence obtain a better performance in usual. Passerini et al. (2004) [16] have recently proposed a decoding strategy based on posterior probabilities. However, according to the analyses of this paper, Passerini et al.'s (2004) [16] method suffers some defects and result in bias. To overcome that, we proposed a variation of it by refining the decomposition process of probability to get smoother estimates. Our bias–variance analysis shows that the decrease in error by our variant is due to a decrease in variance. Besides, we extended an efficient method of obtaining posterior probabilities based on the linear rule for decoding process in Binary ECOC to Ternary ECOC. On ten benchmark datasets, we observe that the two decoding strategies based on posterior probabilities in this paper obtain better performance than other ones in earlier references.     

A novel algorithmic approach to the integration of posterior knowledge into condition-monitoring systems

This paper considers the problem of the integration of 'posterior knowledge' into condition-monitoring systems from both the theoretical and practical points of view. The work is presented in the context of aircraft engine maintenance. A methodology for updating posterior probabilities is proposed for cases where fault conditions are rejected or retained on the basis of external knowledge supplied by an end user, that is the posterior knowledge. A possible fault class ranking is generated following the specification of fault class posterior probability functions. Context-free simulations are used to show the effect of posterior knowledge as part of a maintenance strategy. The simulations are independent of any specific condition-monitoring situation. Preliminary results indicate that posterior knowledge reduces the number of subunit inspections required for isolation of all faults. This has the potential to result in real maintenance cost savings.     

融合样本相似性与HMMs的字符识别技术

样本相似性是两个样本是否属于同一类别的重要依据,而传统的隐马尔可夫建模(HMMs)方法仅根据后验概率进行分类。将二者结合起来,提出一种基于样本相似性的HMMs后验概率调整方法。在该方法中采用距离来描述样本间的相似性,利用规范化的距离相似性度量对后验概率进行适当的调整。在一个单分类器中充分利用了两种分类信息,因此将其用于脱机手写大写金额的识别过程中,取得了良好的效果:在识别精度提高的同时,识别速度变化很小。     

An extension of statistical decision theory with information theoretic cost functions to decision fusion: Part I

A review of information theory and statistical decision theory has led to the recognition that decisions in statistical decision theory can be interpreted as being determined by the similarity between the distribution of probabilities obtained from measurements and characteristic distributions of probabilities representing the members of the set of decisions. The obscure interpretation was found during a review of statistical decision theory for the special case where the cost function of statistical decision theory is an information theoretic cost function.Additional research has found that the resulting information theoretic decision rule has a number of interesting characteristics that may have previously been recognized in terms of mathematical interest, but until now have not been recognized for their implications for information fusion. Bayesian probability theory has been criticized for problematic changes in decisions when hypotheses and decisions are reorganized to different levels of abstraction, weak justification for the selection of prior probabilities, and the need for all probability density functions to be defined. The characteristics of the information theoretic decision rule show that the decisions are less sensitive to changes in the reorganization of hypothesis and decision sets to different levels of abstraction in comparison to Bayesian probability theory. Extension of the information theoretic rule to a fusion rule (to be provided in a companion paper) will be shown to provide increased justification for the selection of prior probabilities through the adoption of Laplace’s principle of indifference. The criticism of the need for all probability density functions can be partially mitigated by arguing that the hypothesis abstraction levels can be selected so that all the probability density functions may be obtained. Further refutation of the third criticism will require that the assumption that the probability density functions are not definitively known but may be ambiguous as well and will not be pursued as a line of inquiry within the two companion papers.     

基于可达概率区间的不确定决策树

针对不确定数据的概率分布难以获取的客观实际,讨论了缺失概率分布的值不确定离散对象的决策树。定义了(条件)概率区间,并证明了(条件)概率区间是可达概率区间;基于可达概率区间,定义了(条件)熵区间,并给出了求解(条件)熵区间的上/下界的方法;采用条件熵区间作为属性选择度量,提出了一种新的不确定决策树,将以0-1划分对象的决策树扩展到以概率区间分配对象的决策树,这样不仅可以处理缺失概率分布的值不确定离散对象,也可以处理确定离散对象。通过在基于UCI数据集的不确定数据集上的实验,证实了不确定决策树是有效的。     

AN INTEGRATED FUZZY APPROACH FOR DETERMINING ENGINEERING CHARACTERISTICS IN CONCRETE INDUSTRY

This paper deals with the modeling of conceptual knowledge to capture the major customer requirements effectively and to transform these requirements systematically into the relevant design requirements. Quality Function Deployment (QFD) is a well-known planning and problem-solving tool for translating customer needs (CNs) into the engineering characteristics (ECs) and can be employed for this modeling. In this study, an integrated methodology is presented to rank ECs for implementing QFD in a fuzzy environment. The proposed methodology uses fuzzy weighted average method as a fuzzy group decision making approach to fuse multiple preference rankings for determining the weights of the customer needs. It adopts a fuzzy Analytic Network Process (ANP) approach which enables the consideration of inner dependencies in a cluster as well as the interdependencies between the clusters to determine the importance of ECs. The proposed approach is illustrated through a case study in ready-mixed concrete industry.     

Improving class probability estimates for imbalanced data

Obtaining good probability estimates is imperative for many applications. The increased uncertainty and typically asymmetric costs surrounding rare events increase this need. Experts (and classification systems) often rely on probabilities to inform decisions. However, we demonstrate that class probability estimates obtained via supervised learning in imbalanced scenarios systematically underestimate the probabilities for minority class instances, despite ostensibly good overall calibration. To our knowledge, this problem has not previously been explored. We propose a new metric, the stratified Brier score, to capture class-specific calibration, analogous to the per-class metrics widely used to assess the discriminative performance of classifiers in imbalanced scenarios. We propose a simple, effective method to mitigate the bias of probability estimates for imbalanced data that bags estimators independently calibrated over balanced bootstrap samples. This approach drastically improves performance on the minority instances without greatly affecting overall calibration. We extend our previous work in this direction by providing ample additional empirical evidence for the utility of this strategy, using both support vector machines and boosted decision trees as base learners. Finally, we show that additional uncertainty can be exploited via a Bayesian approach by considering posterior distributions over bagged probability estimates.     

A preference aggregation method through the estimation of utility intervals

In this paper, a preference aggregation method is developed for ranking alternative courses of actions by combining preference rankings of alternatives given on individual criteria or by individual decision makers. In the method, preference rankings are viewed as constraints on alternative utilities, which are normalized, and linear programming models are constructed to estimate utility intervals, which are weighted and averaged to generate an aggregated utility interval. A simple yet pragmatic interval ranking method is used to compare and/or rank alternatives. The final ranking is generated as the most likely ranking with certain degrees of belief. Three numerical examples are examined to illustrate the potential applications of the proposed method.Scope and purposeThe aggregation of preference rankings has wide applications in group decision making, social choice, committee election and voting systems. The purpose of this paper is to develop a preference aggregation method through the estimation of utility intervals, in which preference rankings are associated with utility intervals that are estimated using linear programming models and aggregated using the simple additive weighting method.