Boosting neural networks

Boosting is a general method for improving the performance of learning algorithms. A recently proposed boosting algorithm, AdaBoost, has been applied with great success to several benchmark machine learning problems using mainly decision trees as base classifiers. In this article we investigate whether AdaBoost also works as well with neural networks, and we discuss the advantages and drawbacks of different versions of the AdaBoost algorithm. In particular, we compare training methods based on sampling the training set and weighting the cost function. The results suggest that random resampling of the training data is not the main explanation of the success of the improvements brought by AdaBoost. This is in contrast to bagging, which directly aims at reducing variance and for which random resampling is essential to obtain the reduction in generalization error. Our system achieves about 1.4% error on a data set of on-line handwritten digits from more than 200 writers. A boosted multilayer network achieved 1.5% error on the UCI letters and 8.1% error on the UCI satellite data set, which is significantly better than boosted decision trees.     

Computing leximin-optimal solutions in constraint networks

In many real-world multiobjective optimization problems one needs to find solutions or alternatives that provide a fair compromise between different conflicting objective functions—which could be criteria in a multicriteria context, or agent utilities in a multiagent context—while being efficient (i.e. informally, ensuring the greatest possible overall agents' satisfaction). This is typically the case in problems implying human agents, where fairness and efficiency requirements must be met. Preference handling, resource allocation problems are another examples of the need for balanced compromises between several conflicting objectives. A way to characterize good solutions in such problems is to use the leximin preorder to compare the vectors of objective values, and to select the solutions which maximize this preorder. In this article, we describe five algorithms for finding leximin-optimal solutions using constraint programming. Three of these algorithms are original. Other ones are adapted, in constraint programming settings, from existing works. The algorithms are compared experimentally on three benchmark problems.     

Boosting the performance of Myrinet networks

Networks of workstations (NOWs) are becoming increasingly popular as a cost-effective alternative to parallel computers. These networks allow the customer to connect processors using irregular topologies, providing the wiring flexibility, scalability, and incremental expansion capability required in this environment. Some of these networks use source routing and wormhole switching. In particular, we are interested in Myrinet networks because it is a well-known commercial product and its behavior can be controlled by the software running in network interfaces (Myrinet Control Program, MCP). Usually, the Myrinet network uses up*/down* routing for computing the paths for every source-destination pair. We propose the In-Transit Buffer (ITB) mechanism to improve network performance. We apply the ITB mechanism to NOWs with up*/down* source routing, like Myrinet, analyzing its behavior on both networks with regular and irregular topologies. The proposed scheme can be implemented on Myrinet networks by only modifying the MCP, without changing the network hardware. We evaluate by simulation several networks with different traffic patterns using timing parameters taken from the Myrinet network. Results show that the current routing schemes used in Myrinet networks can be strongly improved by applying the ITB mechanism. In general, our proposed scheme is able to double the network throughput on medium and large NOWs. Finally, we present a first implementation of the ITB mechanism on a Myrinet network.     

Boosting the performance of Myrinet networks

Networks of workstations (NOWs) are becoming increasingly popular as a cost-effective alternative to parallel computers. These networks allow the customer to connect processors using irregular topologies, providing the wiring flexibility, scalability and incremental expansion capability required in this environment. Some of these networks use source routing and wormhole switching. In particular, we are interested in Myrinet networks because they are a well-known commercial product and their behavior can be controlled by the software running on the network interfaces (the Myrinet Control Program, MCP). Usually, the Myrinet network uses up*/down* routing for computing the paths for every source-destination pair. In this paper, we propose an in-transit buffer (ITB) mechanism to improve the network performance. We apply the ITB mechanism to NOWs with up*/down* source routing, like the Myrinet, analyzing its behavior on networks with both regular and irregular topologies. The proposed scheme can be implemented on Myrinet networks by simply modifying the MCP, without changing the network hardware. We evaluate by simulation several networks with different traffic patterns using timing parameters taken from the Myrinet network. The results show that the current routing schemes used in Myrinet networks can be strongly improved by applying the ITB mechanism. In general, our proposed scheme is able to double the network throughput on medium and large NOWs. Finally, we present a first implementation of the ITB mechanism on a Myrinet network     

Solving multi-granularity temporal constraint networks

Many problems in scheduling, planning, and natural language understanding have been formulated in terms of temporal constraint satisfaction problems (TCSP). These problems have been extensively investigated in the AI literature providing effective solutions for some fragments of the general model. Independently, there has been an effort in the data and knowledge management research community for the formalization of the concept of time granularity and for its applications. This paper considers a framework for integrating the notion of time granularity into TCSP, and investigates the problems of consistency and network solution, which, in this context, involve complex manipulation of the periodic sets representing time granularities. A sound and complete algorithm for consistency checking and for deriving a solution is presented. The paper also investigates the algorithm's computational complexity and several optimization techniques specific to the multi-granularity context. An application to e-commerce workflows illustrates the benefits of the framework and the need for specific reasoning tools.     

Computing weight constraint reachability in large networks

Reachability is a fundamental problem on large-scale networks emerging nowadays in various application domains, such as social networks, communication networks, biological networks, road networks, etc. It has been studied extensively. However, little existing work has studied reachability with realistic constraints imposed on graphs with real-valued edge or node weights. In fact, such weights are very common in many real-world networks, for example, the bandwidth of a link in communication networks, the reliability of an interaction between two proteins in PPI networks, and the handling capacity of a warehouse/storage point in a distribution network. In this paper, we formalize a new yet important reachability query in weighted undirected graphs, called weight constraint reachability (WCR) query that asks: is there a path between nodes \(a\) and \(b\), on which each real-valued edge (or node) weight satisfies a range constraint. We discover an interesting property of WCR, based on which, we design a novel edge-based index structure to answer the WCR query in \(O(1)\) time. Furthermore, we consider the case when the index cannot entirely fit in the memory, which can be very common for emerging massive networks. An I/O-efficient index is proposed, which provides constant I/O (precisely four I/Os) query time with \(O(|V|\log |V|)\) disk-based index size. Extensive experimental studies on both real and synthetic datasets demonstrate the efficiency and scalability of our solutions in answering the WCR query.     

Robust design based on constraint networks

This paper describes a robust design method using constraint networks. As opposed to the traditional statistical robust methodology, the proposed method gives a valid model to analyze parameter uncertainties so as to predict conflicts in concurrent design. The mathematical model, which reflects the requirements of robust design, is given in the paper. A general consistency algorithm is designed using interval arithmetic to refine the intervals. This paper also proves that the consistency algorithm is arc consistent if the constraint network is integrated. The constraint network uses the consistency algorithm to verify the design process early in the process and to assist the designers in determining design variables to reduce the multidisciplinary iterations in concurrent design. The quantitative effect of downstream constraints can be analyzed before determining design parameters and potential conflicts can be predicted. A layout design example shows the validity of the method.     

Neural networks for finite constraint satisfaction

Constraint Satisfaction Problems (CSPs) are in general NP-hard, and a general deterministic polynomial time algorithm is not known. They play a central role in real-life problems. The satisfaction of a Conjunctive Normal Form (CNF-SAT)is the core of any CSP. We present a new modelisation technique for any CSP with finite variable domains, and, in particular, for solving CNF-SAT. The knowledge representation is based on two fundamental types of constraint: the choice constraint, and the exclusion constraint. These models are then implemented by means of several different neural networks, some based on backpropagation learning and others on different procedures. All these networks are trained through a supervised procedure, and learn to efficiently solve CNF-SAT. The results of significant tests are described: they show that some networks can effectively solve the proposed problems.     

Boosting the accuracy of differentially private in weighted social networks

Multimedia Tools and Applications - Social network not only helps people to build its internet applicable service, but also collects a large amount of user information (i.e., sensitive data), which...     

Stereopsis by constraint learning feed-forward neural networks

A neural network (NN) approach to the problem of steropsis is presented. The correspondence problem (finding the correct matches between pixels of the epipolar lines of the stereo pair from among all the possible matches) is posed as a noniterative many-to-one mapping. Two multilayer feedforward NNs are utilized to learn and code this nonlinear and complex mapping using the backpropagation learning rule and a training set. The first NN is a conventional fully connected net while the second is a sparsely connected NN with a fixed number of hidden layer nodes. All the applicable constraints are learned and internally coded by the NNs enabling them to be more flexible and more accurate than previous methods. The approach is successfully tested on several random-dot stereograms. It is shown that the nets can generalize their learned mappings to cases outside their training sets and to noisy images. Advantages over the Marr-Poggio algorithm are discussed, and it is shown that the NNs performances are superior.     

Bayes networks for estimating the number of solutions of constraint networks

The problem of counting the number of solutions to a constraint network (CN) (also called constraint satisfaction problems, CSPs) is rephrased in terms of probability updating in Bayes networks. Approximating the probabilities in Bayes networks is a problem which has been studied for a while, and may well provide a good approximation to counting the number of solutions. We use a simple approximation based on independence, and show that it is correct for tree‐structured CNs. For other CNs, it is less optimistic than a spanning‐tree approximation suggested in prior work. Experiments show that the Bayes nets estimation is more accurate on the average, compared to competing methods (the spanning‐tree, as well as a scheme based on a product of all compatible pairs of values). We present empirical evidence that our approximation may also be a useful (value ordering) search heuristic for finding a single solution to a constraint satisfaction problem. This revised version was published online in June 2006 with corrections to the Cover Date.     

Maximizing influence under influence loss constraint in social networks

Influence maximization is a fundamental research problem in social networks. Viral marketing, one of its applications, aims to select a small set of users to adopt a product, so that the word-of-mouth effect can subsequently trigger a large cascade of further adoption in social networks. The problem of influence maximization is to select a set of K nodes from a social network so that the spread of influence is maximized over the network. Previous research on mining top-K influential nodes assumes that all of the selected K nodes can propagate the influence as expected. However, some of the selected nodes may not function well in practice, which leads to influence loss of top-K nodes. In this paper, we study an alternative influence maximization problem which is naturally motivated by the reliability constraint of nodes in social networks. We aim to find top-K influential nodes given a threshold of influence loss due to the failure of a subset of R(<K) nodes. To solve the new type of influence maximization problem, we propose an approach based on constrained simulated annealing and further improve its performance through efficiently estimating the influence loss. We provide experimental results over multiple real-world social networks in support. This research will further support practical applications of social networks in various domains particularly where reliability would be a main concern in a system deployment.     

Hybrid backtracking bounded by tree-decomposition of constraint networks

We propose a framework for solving CSPs based both on backtracking techniques and on the notion of tree-decomposition of the constraint networks. This mixed approach permits us to define a new framework for the enumeration, which we expect that it will benefit from the advantages of two approaches: a practical efficiency of enumerative algorithms and a warranty of a limited time complexity by an approximation of the tree-width of the constraint networks. Finally, experimental results allow us to show the advantages of this approach.     

#br# 一种面向枚举类型的配置约束提取方法

有研究表明软件配置故障已成为导致计算机系统异常和崩溃的一个重要因素。配置故障是由于用户无法充分地获取配置约束的信息进行误配置造成的,由于用户缺乏软件领域知识,配置故障难以避免。因此,如何对软件配置项的约束条件进行精确的分析和提取,从而为软件配置的故障诊断与修复提供依据,具有重要的研究意义。具体而言,枚举类型作为软件系统的常用类型,其取值空间的限制经常导致系统软件配置故障。基于此系统调研了6款常用的C/C++开源软件,包括Apache Httpd、Nginx、Postfix、MySQL、Redis和PostgreSQL的枚举类型配置约束特性,并针对已有方法存在枚举类型配置项取值空间漏报的问题,基于程序分析的方法设计和实现了面向枚举类型配置的自动化配置约束提取方法,大幅提高了针对以上开源软件的配置约束提取准确率,提升了软件配置的可用性和配置故障诊断能力。     

Causal temporal constraint networks for representing temporal knowledge

In this work we describe causal temporal constraint networks (CTCN) as a new computable model for representing temporal information and efficiently handling causality. The proposed model enables qualitative and quantitative temporal constraints to be established, introduces the representation of causal constraints, and suggests mechanisms for representing inexact temporal knowledge. The temporal handling of information is achieved by structuring the information in different interpretation contexts, linked to each other through an inference mechanism which obtains interpretations that are consistent with the original temporal information. In carrying out inferences, we take into account the temporal relationships between events, the possible inexactitude associated with the events, and the atemporal or static information which affects the interpretation pattern being considered. The proposed schema is illustrated with an application developed using the CommonKADS methodology.     

Data censoring with network lifetime constraint in wireless sensor networks

In wireless sensor networks (WSNs), senor nodes are usually battery-powered with limited energy budget. The network lifetime is directly related to the energy consumption of each node. Online censoring is an effective approach to reduce the overall energy consumption by only transmitting statistical informative data. However, the network lifetime is not proportionally extended with online censoring, since individual sensor may still suffer from energy shortage due to frequent transmission of informative data or transmission over long distance. In this paper, a parameters estimation problem is considered in WSNs, where the goal is to minimize the estimation error under the network lifetime constraint. Two censoring algorithms are developed, which allow sensor nodes to make decisions locally on whether to transmit the sampled data. The proposed algorithms can extend the network lifetime with little performance loss. Simulation results validate their effectivenesses.     

A lagrangian propagator for artificial neural networks in constraint programming

This paper discusses a new method to perform propagation over a (two-layer, feed-forward) Neural Network embedded in a Constraint Programming model. The method is meant to be employed in Empirical Model Learning, a technique designed to enable optimal decision making over systems that cannot be modeled via conventional declarative means. The key step in Empirical Model Learning is to embed a Machine Learning model into a combinatorial model. It has been showed that Neural Networks can be embedded in a Constraint Programming model by simply encoding each neuron as a global constraint, which is then propagated individually. Unfortunately, this decomposition approach may lead to weak bounds. To overcome such limitation, we propose a new network-level propagator based on a non-linear Lagrangian relaxation that is solved with a subgradient algorithm. The method proved capable of dramatically reducing the search tree size on a thermal-aware dispatching problem on multicore CPUs. The overhead for optimizing the Lagrangian multipliers is kept within a reasonable level via a few simple techniques. This paper is an extended version of [27], featuring an improved structure, a new filtering technique for the network inputs, a set of overhead reduction techniques, and a thorough experimentation.