Effective sparsity control in deep belief networks using normal regularization term

Nowadays the use of deep network architectures has become widespread in machine learning. Deep belief networks (DBNs) have deep network architectures to create a powerful generative model using training data. Deep belief networks can be used in classification and feature learning. A DBN can be learned unsupervised, and then the learned features are suitable for a simple classifier (like a linear classifier) with a few labeled data. In addition, according to researches, by using sparsity in DBNs we can learn useful low-level feature representations for unlabeled data. In sparse representation, we have the property that learned features can be interpreted, i.e., correspond to meaningful aspects of the input, and capture factors of variation in the data. Different methods are proposed to build sparse DBNs. In this paper, we proposed a new method that has different behavior according to deviation of the activation of the hidden units from a (low) fixed value. In addition, our proposed regularization term has a variance parameter that can control the force degree of sparseness. According to the results, our new method achieves the best recognition accuracy on the test sets in different datasets with different applications (image, speech and text) and we can achieve incredible results when using a different number of training samples, especially when we have a few samples for training.     

Knowledge distilling based model compression and feature learning in fault diagnosis

Recently, there has been interest in developing diagnosis methods that combine model-based and data-driven diagnosis. In both approaches, selecting the relevant measurements or extracting important features from historical data is a key determiner of the success of the algorithm. Recently, deep learning methods have been effective in automating the feature selection process. Autoencoders have been shown to be an effective neural network configuration for extracting features from complex data, however, they may also learn irrelevant features. In addition, end-to-end classification neural networks have also been used for diagnosis, but like autoencoders, this method may also learn unimportant features thus making the diagnostic inference scheme inefficient. To rapidly extract significant fault features, this paper employs end-to-end networks and develops a new feature extraction method based on importance analysis and knowledge distilling. First, a set of cumbersome neural network models are trained to predict faults and some of their internal values are defined as features. Then an occlusion-based importance analysis method is developed to select the most relevant input variables and learned features. Finally, a simple student neural network model is designed based on the previous analysis results and an improved knowledge distilling method is proposed to train the student model. Because of the way the cumbersome networks are trained, only fault features are learned, with the importance analysis further pruning the relevant feature set. These features can be rapidly generated by the student model. We discuss the algorithms, and then apply our method to two typical dynamic systems, a communication system and a 10-tank system employed to demonstrate the proposed approach.     

基于深度信念网络的Android恶意应用检测方法

传统的机器学习算法无法有效地从海量的行为特征中选择出有本质的行为特征来对未知的Android恶意应用进行检测。为了解决这个问题,提出DBNSel,一种基于深度信念网络模型的Android恶意应用检测方法。为了实现该方法,首先通过静态分析方法从Android应用中提取5类不同的属性。其次,建立深度信念网络模型从提取到的属性中进行选择和学习。最后,使用学习到的属性来对未知类型的Android恶意应用进行检测。在实验阶段,使用一个由3 986个Android正常应用和3 986个Android恶意应用组成的数据集来验证DBNSel的有效性。实验结果表明,DBNSel的检测结果要优于其他几种已有的检测方法,并可以达到99.4%的检测准确率。此外,DBNSel具有较低的运行开销,可以适应于更大规模的真实环境下的Android恶意应用检测。     

Bayesian update of dialogue state: A POMDP framework for spoken dialogue systems

This paper describes a statistically motivated framework for performing real-time dialogue state updates and policy learning in a spoken dialogue system. The framework is based on the partially observable Markov decision process (POMDP), which provides a well-founded, statistical model of spoken dialogue management. However, exact belief state updates in a POMDP model are computationally intractable so approximate methods must be used. This paper presents a tractable method based on the loopy belief propagation algorithm. Various simplifications are made, which improve the efficiency significantly compared to the original algorithm as well as compared to other POMDP-based dialogue state updating approaches. A second contribution of this paper is a method for learning in spoken dialogue systems which uses a component-based policy with the episodic Natural Actor Critic algorithm.The framework proposed in this paper was tested on both simulations and in a user trial. Both indicated that using Bayesian updates of the dialogue state significantly outperforms traditional definitions of the dialogue state. Policy learning worked effectively and the learned policy outperformed all others on simulations. In user trials the learned policy was also competitive, although its optimality was less conclusive. Overall, the Bayesian update of dialogue state framework was shown to be a feasible and effective approach to building real-world POMDP-based dialogue systems.     

Irrelevance reduction with locality-sensitive hash learning for efficient cross-media retrieval

Cross-media retrieval is an imperative approach to handle the explosive growth of multimodal data on the web. However, existing approaches to cross-media retrieval are computationally expensive due to high dimensionality. To efficiently retrieve in multimodal data, it is essential to reduce the proportion of irrelevant documents. In this paper, we propose a fast cross-media retrieval approach (FCMR) based on locality-sensitive hashing (LSH) and neural networks. One modality of multimodal information is projected by LSH algorithm to cluster similar objects into the same hash bucket and dissimilar objects into different ones and then another modality is mapped into these hash buckets using hash functions learned through neural networks. Once given a textual or visual query, it can be efficiently mapped to a hash bucket in which objects stored can be near neighbors of this query. Experimental results show that, in the set of the queries’ near neighbors obtained by the proposed method, the proportions of relevant documents can be much boosted, and it indicates that the retrieval based on near neighbors can be effectively conducted. Further evaluations on two public datasets demonstrate the efficacy of the proposed retrieval method compared to the baselines.     

基于最短描述长度的序列图像运动分割

提出了一种分两个阶段的运动估计和分割方法。首先采用小块对图像做传统全局搜索的块匹配运动估计,得到每块相应搜索范围内的误差图,根据误差图判决相邻块运动的一致性将一致性好的块合并成一个区域并同时得到区域的运动矢量。进一步的区域合并在最短描述长度准则指导下进行,每对区域合并的依据是合并后新区域的描述长度小于合并前两区域描述长度之间,如此迭代合并直至所有相邻区域均不满足条件。     

A FRAMEWORK FOR LOGICS OF EXPLICIT BELIEF

The epistemic notions of knowledge and belief have most commonly been modeled by means of possible worlds semantics. In such approaches an agent knows (or believes) all logical consequences of its beliefs. Consequently, several approaches have been proposed to model systems of explicit belief, more suited to modeling finite agents or computers. In this paper a general framework is developed for the specification of logics of explicit belief. A generalization of possible worlds, called situations, is adopted. However the notion of an accessibility relation is not employed; instead a sentence is believed if the explicit proposition expressed by the sentence appears among a set of propositions associated with an agent at a situation. Since explicit propositions may be taken as corresponding to "belief contexts" or "frames of mind," the framework also provides a setting for investigating such approaches to belief. The approach provides a uniform and flexible basis from which various issues of explicit belief may be addressed and from which systems may be contrasted and compared. A family of logics is developed using this framework, which extends previous approaches and addresses issues raised by these earlier approaches. The more interesting of these logics are tractable, in that determining if a belief follows from a set of beliefs, given certain assumptions, can be accomplished in polynomial time.     

Pedestrian trajectory prediction using BiRNN encoder–decoder framework*

Autonomous mobile robots navigating through human crowds are required to foresee the future trajectories of surrounding pedestrians and accordingly plan safe paths to avoid any possible collision. This paper presents a novel approach for pedestrian trajectory prediction. In particular, we developed a new method based on an encoder–decoder framework using bidirectional recurrent neural networks (BiRNN). The difficulty of incorporating social interactions into the model has been addressed thanks to the special structure of BiRNN enhanced by the attention mechanism, a proximity-independent model of the relative importance of each pedestrian. The main difference between our and the previous approaches is that BiRNN allows us to employs information on the future state of the pedestrians. We tested the performance of our method on several public datasets. The proposed model outperforms the current state-of-the-art approaches on most of these datasets. Furthermore, we analyze the resulting predicted trajectories and the learned attention scores to prove the advantages of BiRRNs on recognizing social interactions.     

Belief Revision as Applied within a Descriptive Model of Jury Deliberations

Belief revision is a well-researched topic within Artificial Intelligence (AI). We argue that the new model of belief revision as discussed here is suitable for general modelling of judicial decision making, along with the extant approach as known from jury research. The new approach to belief revision is of general interest, whenever attitudes to information are to be simulated within a multi-agent environment with agents holding local beliefs yet by interacting with, and influencing, other agents who are deliberating collectively. The principle of 'priority to the incoming information', as known from AI models of belief revision, is problematic when applied to factfinding by a jury. The present approach incorporates a computable model for local belief revision, such that a principle of recoverability is adopted. By this principle, any previously held belief must belong to the current cognitive state if consistent with it. For the purposes of jury simulation such a model calls for refinement. Yet, we claim, it constitutes a valid basis for an open system where other AI functionalities (or outer stimuli) could attempt to handle other aspects of the deliberation which are more specific to legal narratives, to argumentation in court, and then to the debate among the jurors.     

Iterated belief change in the situation calculus

John McCarthy's situation calculus has left an enduring mark on artificial intelligence research. This simple yet elegant formalism for modelling and reasoning about dynamic systems is still in common use more than forty years since it was first proposed. The ability to reason about action and change has long been considered a necessary component for any intelligent system. The situation calculus and its numerous extensions as well as the many competing proposals that it has inspired deal with this problem to some extent. In this paper, we offer a new approach to belief change associated with performing actions that addresses some of the shortcomings of these approaches. In particular, our approach is based on a well-developed theory of action in the situation calculus extended to deal with belief. Moreover, by augmenting this approach with a notion of plausibility over situations, our account handles nested belief, belief introspection, mistaken belief, and handles belief revision and belief update together with iterated belief change.     

A GAs based approach for mining breast cancer pattern

Data mining usually means the methodologies and tools for the efficient new knowledge discovery from databases. In this paper, a genetic algorithms (GAs) based approach to assess breast cancer pattern is proposed for extracting the decision rules including the predictors, the corresponding inequality and threshold values simultaneously so as to building a decision-making model with maximum prediction accuracy. Early many studies of handling the breast cancer diagnostic problems used the statistical related techniques. As the diagnosis of breast cancer is highly nonlinear in nature, it is hard to develop a comprehensive model taking into account all the independent variables using conventional statistical approaches. Recently, numerous studies have demonstrated that neural networks (NNs) are more reliable than the traditional statistical approaches and the dynamic stress method. The usefulness of using NNs have been reported in literatures but the most obstacle is the in the building and using the model in which the classification rules are hard to be realized. We compared our results against a commercial data mining software, and we show experimentally that the proposed rule extraction approach is promising for improving prediction accuracy and enhancing the modeling simplicity. In particular, our approach is capable of extracting rules which can be developed as a computer model for prediction or classification of breast cancer potential like expert systems.     

Bayesian belief network learning algorithms for modeling contextual relationships in natural imagery: a comparative study

This paper presents a comparative study of Bayesian belief network structure learning algorithms with a view to identify a suitable algorithm for modeling the contextual relations among objects typically found in natural imagery. Four popular structure learning algorithms are compared: two constraint-based algorithms (PC proposed by Spirtes and Glymour and Fast Incremental Association Markov Blanket proposed by Yaramakala and Margaritis), a score-based algorithm (Hill Climbing as implemented by Daly), and a hybrid algorithm (Max-Min Hill Climbing proposed by Tsamardinos et al.). Contrary to the belief regarding the superiority of constraint-based approaches, our empirical results show that a score-based approach performs better on our context dataset in terms of prediction power and learning time. The hybrid algorithm could achieve similar prediction performance as the score-based approach, but requires longer time to learn the desired network. Another interesting fact the study has revealed is the existence of strong correspondence between the linear correlation pattern within the dataset and the edges found in the learned networks.     

Estimating body shape of dressed humans

The paper presents a method to estimate the detailed 3D body shape of a person even if heavy or loose clothing is worn. The approach is based on a space of human shapes, learned from a large database of registered body scans. Together with this database we use as input a 3D scan or model of the person wearing clothes and apply a fitting method, based on ICP (iterated closest point) registration and Laplacian mesh deformation. The statistical model of human body shapes enforces that the model stays within the space of human shapes. The method therefore allows us to compute the most likely shape and pose of the subject, even if it is heavily occluded or body parts are not visible. Several experiments demonstrate the applicability and accuracy of our approach to recover occluded or missing body parts from 3D laser scans.     

Adaptive sparseness for supervised learning

The goal of supervised learning is to infer a functional mapping based on a set of training examples. To achieve good generalization, it is necessary to control the "complexity" of the learned function. In Bayesian approaches, this is done by adopting a prior for the parameters of the function being learned. We propose a Bayesian approach to supervised learning, which leads to sparse solutions; that is, in which irrelevant parameters are automatically set exactly to zero. Other ways to obtain sparse classifiers (such as Laplacian priors, support vector machines) involve (hyper)parameters which control the degree of sparseness of the resulting classifiers; these parameters have to be somehow adjusted/estimated from the training data. In contrast, our approach does not involve any (hyper)parameters to be adjusted or estimated. This is achieved by a hierarchical-Bayes interpretation of the Laplacian prior, which is then modified by the adoption of a Jeffreys' noninformative hyperprior. Implementation is carried out by an expectation-maximization (EM) algorithm. Experiments with several benchmark data sets show that the proposed approach yields state-of-the-art performance. In particular, our method outperforms SVMs and performs competitively with the best alternative techniques, although it involves no tuning or adjustment of sparseness-controlling hyperparameters.     

Personalized Tag Recommendation Using Social Influence

Tag recommendation encourages users to add more tags in bridging the semantic gap between human concept and the features of media object,which provides a feasible solution for content-based multimedia information retrieval.In this paper,we study personalized tag recommendation in a popular online photo sharing site - Flickr.Social relationship information of users is collected to generate an online social network.From the perspective of network topology,we propose node topological potential to characterize user’s social influence.With this metric,we distinguish different social relations between users and find out those who really have influence on the target users.Tag recommendations are based on tagging history and the latent personalized preference learned from those who have most influence in user’s social network.We evaluate our method on large scale real-world data.The experimental results demonstrate that our method can outperform the non-personalized global co-occurrence method and other two state-of-the-art personalized approaches using social networks.We also analyze the further usage of our approach for the cold-start problem of tag recommendation.