Chaotic simulated annealing with decaying chaotic noise

By adding chaotic noise to each neuron of the discrete-time continuous-output Hopfield neural network (HNN) and gradually reducing the noise, a chaotic neural network is proposed so that it is initially chaotic but eventually convergent, and, thus, has richer and more flexible dynamics compared to the HNN. The proposed network is applied to the traveling salesman problem (TSP) and that results are highly satisfactory. That is, the transient chaos enables the network to escape from local energy minima and to find global minima in 100% of the simulations for four-city and ten-city TSPs, as well as near-optimal solutions in most of runs for a 48-city TSP.     

基于局部进化的Hopfield神经网络的优化计算方法

提出一种基于局部进化的Hopfield神经网络优化计算方法，该方法将遗传算法和Hopfield神经网络结合在一起，克服了Hopfield神经网络易收敛到局部最优值的缺点，以及遗传算法收敛速度慢的缺点。该方法首先由Hopfield神经网络进行状态方程的迭代计算降低网络能量，收敛后的Hopfield神经网络在局部范围内进行遗传算法寻优，以跳出可能的局部最优值陷阱，再由Hopfield神经网络进一步迭代优化。这种局部进化的Hopfield神经网络优化计算方法尤其适合于大规模的优化问题，对图像分割问题和规模较大的200城市旅行商问题的优化计算结果表明，其全局收敛率和收敛速度明显提高。     

一种基于聚类技术的Hopfield网络求解TSP方法

Hopfiled神经网络方法已被广泛用于求解旅行商问题（TSP），但对于解中规模和大规模的TSP，存在效果不理想甚至难以求解的问题。为了较好地解决这个问题，该文提出一种K-Means聚类算法与Hopfield网络方法相结合求解TSP的新方法，先应用聚类算法对所给城市进行聚类以获得几组规模较小的城市，然后对每一组城市应用Hopfield网络方法进行求解，最后把求解后的每组城市连接起来。计算机仿真结果表明，该方法可以获得最优有效解，并且解的质量明显提高，对求解中大规模的TSP比较有效。     

An argument for abandoning the travelling salesman problem as a neural-network benchmark.

In this paper, a distinction is drawn between research which assesses the suitability of the Hopfield network for solving the travelling salesman problem (TSP) and research which attempts to determine the effectiveness of the Hopfield network as an optimization technique. It is argued that the TSP is generally misused as a benchmark for the latter goal, with the existence of an alternative linear formulation giving rise to unreasonable comparisons.     

利用Hopfield神经网络求解旅行商问题研究

本文主要研究利用连续的Hopfield网络求解TSP问题,从连续的Hopfield神经网络原理出发,结合TSP问题的要求,在给定参数要求下求得问题的最优解。并分析了实际算法的弱点,给出分析改进算法,加快了算法的收敛速度,改善有效解并提高最优解的比例。     

Scheduling multiprocessor job with resource and timing constraintsusing neural networks

The Hopfield neural network is extensively applied to obtaining an optimal/feasible solution in many different applications such as the traveling salesman problem (TSP), a typical discrete combinatorial problem. Although providing rapid convergence to the solution, TSP frequently converges to a local minimum. Stochastic simulated annealing is a highly effective means of obtaining an optimal solution capable of preventing the local minimum. This important feature is embedded into a Hopfield neural network to derive a new technique, i.e., mean field annealing. This work applies the Hopfield neural network and the normalized mean field annealing technique, respectively, to resolve a multiprocessor problem (known to be a NP-hard problem) with no process migration, constrained times (execution time and deadline) and limited resources. Simulation results demonstrate that the derived energy function works effectively for this class of problems.     

A columnar competitive model for solving combinatorial optimization problems

The major drawbacks of the Hopfield network when it is applied to some combinatorial problems, e.g., the traveling salesman problem (TSP), are invalidity of the obtained solutions, trial-and-error setting value process of the network parameters and low-computation efficiency. This letter presents a columnar competitive model (CCM) which incorporates winner-takes-all (WTA) learning rule for solving the TSP. Theoretical analysis for the convergence of the CCM shows that the competitive computational neural network guarantees the convergence to valid states and avoids the onerous procedures of determining the penalty parameters. In addition, its intrinsic competitive learning mechanism enables a fast and effective evolving of the network. The simulation results illustrate that the competitive model offers more and better valid solutions as compared to the original Hopfield network.     

Managing spatio-temporal complexity in Hopfield neural network simulations for large-scale static optimization

A simulation methodology, which trades space complexity with time complexity, to create the Hopfield neural network weight matrix, the costliest data structure for simulation of Hopfield neural network algorithm for large-scale optimization problems, is proposed. Modular composition of a weight term of the Hopfield neural network weight matrix for a generic static optimization problem, which facilitates construction and reconstruction of the weights on demand during a simulation, is exposed. Proposed methodology is demonstrated on a static combinatorial optimization problem, namely the Traveling Salesman Problem (TSP), through the algebraic procedure for temporal (versus spatial) weight matrix construction, pseudo code and C/C++ code implementation, and an associated simulation study. The proposed methodology is successfully tested through simulation on a general purpose Windows™-AMD™ platform for up to 1000 city Traveling Salesman Problem instance, which would require approximately no less than 1TB of memory to be allocated simply to instantiate the weight matrix in the memory space of the simulation process.     

TSP及其基于Hopfield网络优化的研究

Ｈｏｐｆｉｅｌｄ网络（ＨＮＮ）是一种有效的优化模型,但存在易收敛到非法解或局部极小以及对模型参数与初值依赖性强的缺点。旅行商问题（ＴＳＰ）是研究算法性能的典型算例,通过对其进行计算机仿真优化,分析归纳了ＨＮＮ模型存在缺点的原因,总结并提出若干改进方法与思想。同时,针对ＴＳＰ问题的工程背景提出了若干发展性研究内容与方法。     

一种基于退火策略的混沌神经网络优化算法

Ｈｏｐｆｉｅｌｄ网络（ＨＮＮ）中引入混沌机制,首先在混沌动态下粗搜索,并利用退火策略控制混沌动态退出和逆分贫出现,进而ＨＮＮ梯度优化搜索,提出了一种具有随机性和确定性并存的优化算法,对经典旅行商（ＴＳＰ）的研究,表明算法具有很强的克服陷入局部极小能力,较大程度提高了优化、时间和对初值的鲁棒性能,同时给出了模型参数对性能影响的一些结论。     

The road to chaos by time-asymmetric Hebbian learning in recurrent neural networks

This letter aims at studying the impact of iterative Hebbian learning algorithms on the recurrent neural network's underlying dynamics. First, an iterative supervised learning algorithm is discussed. An essential improvement of this algorithm consists of indexing the attractor information items by means of external stimuli rather than by using only initial conditions, as Hopfield originally proposed. Modifying the stimuli mainly results in a change of the entire internal dynamics, leading to an enlargement of the set of attractors and potential memory bags. The impact of the learning on the network's dynamics is the following: the more information to be stored as limit cycle attractors of the neural network, the more chaos prevails as the background dynamical regime of the network. In fact, the background chaos spreads widely and adopts a very unstructured shape similar to white noise. Next, we introduce a new form of supervised learning that is more plausible from a biological point of view: the network has to learn to react to an external stimulus by cycling through a sequence that is no longer specified a priori. Based on its spontaneous dynamics, the network decides "on its own" the dynamical patterns to be associated with the stimuli. Compared with classical supervised learning, huge enhancements in storing capacity and computational cost have been observed. Moreover, this new form of supervised learning, by being more "respectful" of the network intrinsic dynamics, maintains much more structure in the obtained chaos. It is still possible to observe the traces of the learned attractors in the chaotic regime. This complex but still very informative regime is referred to as "frustrated chaos."     

A novel hybrid-maximum neural network in stereo-matching process

In the present paper, the completely innovative architecture of artificial neural network based on Hopfield structure for solving a stereo-matching problem—hybrid neural network, consisting of the classical analog Hopfield neural network and the Maximum Neural Network—is described. The application of this kind of structure as a part of assistive device for visually impaired individuals is considered. The role of the analog Hopfield network is to find the attraction area of the global minimum, whereas Maximum Neural Network is finding accurate location of this minimum. The network presented here is characterized by an extremely high rate of work performance with the same accuracy as a classical Hopfield-like network, which makes it possible to use this kind of structure as a part of systems working in real time. The network considered here underwent experimental tests with the use of real stereo pictures as well as simulated stereo images. This enables error calculation and direct comparison with the classic analog Hopfield neural network as well as other networks proposed in the literature.     

Real time identification and control of dynamic systems using recurrent neural networks

Recently, several recurrent neural networks for solving constraint optimization problems were developed. In this paper, we propose a novel approach to the use of a projection neural network for solving real time identification and control of time varying systems. In addition to low complexity and simple structure, the proposed neural network can solve wider classes of time varying systems compare with other neural networks that are used for optimization such as Hopfield neural networks. Simulation results demonstrate the effectiveness and characteristics of the proposed neural network compared with a Hopfield neural network.     

Neural network approach for solving the maximal common subgraphproblem

A new formulation of the maximal common subgraph problem (MCSP), that is implemented using a two-stage Hopfield neural network, is given. Relative merits of this proposed formulation, with respect to current neural network-based solutions as well as classical sequential-search-based solutions, are discussed.     

Mathematical improvement of the Hopfield model for feasiblesolutions to the traveling salesman problem by a synapse dynamicalsystem

It is well known that the Hopfield Model (HM) for neural networks to solve the Traveling Salesman Problem (TSP) suffers from three major drawbacks. (1) It can converge on nonoptimal locally minimum solutions. (2) It can converge on infeasible solutions. (3) Results are very sensitive to the careful tuning of its parameters. A number of methods have been proposed to overcome (a) well. In contrast, work on (b) and (c) has not been sufficient; techniques have not been generalized to more general optimization problems. Thus this paper mathematically resolves (b) and (c) to such an extent that the resolution can be applied to solving with some general network continuous optimization problems including the Hopfield version of the TSP. It first constructs an Extended HM (E-HM) that overcomes both (b) and (c). Fundamental techniques of the E-HM lie in the addition of a synapse dynamical system cooperated with the current HM unit dynamical system. It is this synapse dynamical system that makes the TSP constraint hold at any final states for whatever choices of the IIM parameters and an initial state. The paper then generalizes the E-HM further to a network that can solve a class of continuous optimization problems with a constraint equation where both of the objective function and the constraint function are nonnegative and continuously differentiable.     

Design and analysis of an efficient neural network model for solving nonlinear optimization problems

This paper presents an efficient approach based on a recurrent neural network for solving constrained nonlinear optimization. More specifically, a modified Hopfield network is developed, and its internal parameters are computed using the valid-subspace technique. These parameters guarantee the convergence of the network to the equilibrium points that represent an optimal feasible solution. The main advantage of the developed network is that it handles optimization and constraint terms in different stages with no interference from each other. Moreover, the proposed approach does not require specification for penalty and weighting parameters for its initialization. A study of the modified Hopfield model is also developed to analyse its stability and convergence. Simulation results are provided to demonstrate the performance of the proposed neural network.     

Solving optimization problems with variable-constraint by anextended Cohen-Grossberg model

A variety of real-world problems can be formulated as continuous optimization problems with variable constraint. It is well-known, however, that it is difficult to develop a unified method for obtaining their feasible solutions. We have recognized that the recent work of solving the traveling salesman problem (TSP) by the Hopfield model explores an innovative approach to them as well as combinatorial optimization problems. The Hopfield model is generalized into the Cohen-Grossberg model (CGM) to which a specific Lyapunov function has been found. This paper thus extends the Hopfield method onto the CGM in order to develop a unified solving-method of continuous optimization problems with variable-constraint. Specifically, we consider a certain class of continuous optimization problems with a constraint equation including the Hopfield version of the TSP as a particular member. Then we theoretically develop a method that, from any given problem of that class, derives a network of an extended CGM to provide feasible solutions to it. The main idea for constructing that extended CGM lies in adding to it a synapse dynamical system concurrently operating with its current unit dynamical system so that the constraint equation can be enforced to satisfaction at final states. This construction is also motivated by previous neuron models in biophysics and learning algorithms in neural networks     

Functional abilities of a stochastic logic neural network

The authors have studied the information processing ability of stochastic logic neural networks, which constitute one of the pulse-coded artificial neural network families. These networks realize pseudoanalog performance with local learning rules using digital circuits, and therefore suit silicon technology. The synaptic weights and the outputs of neurons in stochastic logic are represented by stochastic pulse sequences. The limited range of the synaptic weights reduces the coding noise and suppresses the degradation of memory storage capacity. To study the effect of the coding noise on an optimization problem, the authors simulate a probabilistic Hopfield model (Gaussian machine) which has a continuous neuron output function and probabilistic behavior. A proper choice of the coding noise amplitude and scheduling improves the network's solutions of the traveling salesman problem (TSP). These results suggest that stochastic logic may be useful for implementing probabilistic dynamics as well as deterministic dynamics.     

Hopfield网络求解TSP的算法改进

本文在对Hopfield神经网络求解旅行商(TSP)问题的算法进行研究的基础上结合实例针对典型改进算法的优缺点作了进一步探讨。     

基于连续Hopfield网络求解TSP的新方法

当连续Hopfield网络及其能量函数同时具有自反馈或不具有自反馈时,称之为一致连续Hopfield网络．在分析了一致连续Hopfield网络能量稳定性的基础上,进一步研究了当网络有自反馈,而其能量函数无自反馈的情况下,网络能量变化的性质,分别给出了使能量函数上升、下降和不变的条件．利用这一理论,可以克服由于梯度下降法所导致的网络能量函数总是下降,从而使网络陷入局部极小值或不可行解的现象．最后在这个理论的基础上我们给出了一种新的求解TSP(traveling salesman problem)的方法,仿真研究表明此方法对于求解TSP问题是很有效的．