Nonmonotone Levenberg�CMarquardt training of recurrent neural architectures for processing symbolic sequences

In this paper, we present nonmonotone variants of the Levenberg–Marquardt (LM) method for training recurrent neural networks (RNNs). These methods inherit the benefits of previously developed LM with momentum algorithms and are equipped with nonmonotone criteria, allowing temporal increase in training errors, and an adaptive scheme for tuning the size of the nonmonotone slide window. The proposed algorithms are applied to training RNNs of various sizes and architectures in symbolic sequence-processing problems. Experiments show that the proposed nonmonotone learning algorithms train more effectively RNNs for sequence processing than the original monotone methods.     

Orthogonal Transforms in Bases of Slant Step Functions. II. Methods of Generating Orthogonal Slant Transforms with Fast Algorithms for Image Coding and Compression

Recurrent matrix methods and methods based on the Walsh transform and rotation matrices generating orthogonal slant transforms of high and low correlation are proposed. These methods are used to develop efficient fast slant-transform algorithms without multiplication and additional permutations of output data. Continued from Cybernetics and Systems Analysis, Vol. 41, No. 3, 2005. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 4, pp. 116–132, July–August 2005.     

Sequential Estimation of the Parameters of Regression Model

Modified Gauss and Gram–Schmidt methods are proposed for recursive estimation of the parameters of sequentially complicated structures of regression models in observation-based simulation. Run times of these algorithms are compared with those of the recursive algorithm based on the well-known bordering method. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 4, pp. 184–187, July–August 2005.     

Parallel nonlinear preconditioners on multicore architectures

Parallel nonlinear preconditioners, for solving mildly nonlinear systems, are proposed. These algorithms are based on both the Fletcher–Reeves version of the nonlinear conjugate gradient method and a polynomial preconditioner type based on block two-stage methods. The behavior of these algorithms is analyzed when incomplete LU factorizations are used in order to obtain the inner splittings of the block two-stage method. As our illustrative example we have considered a nonlinear elliptic partial differential equation, known as the Bratu problem. The reported experiments show the performance of the algorithms designed in this work on two multicore architectures.     

Parallel randomized sampling for support vector machine (SVM) and support vector regression (SVR)

A parallel randomized support vector machine (PRSVM) and a parallel randomized support vector regression (PRSVR) algorithm based on a randomized sampling technique are proposed in this paper. The proposed PRSVM and PRSVR have four major advantages over previous methods. (1) We prove that the proposed algorithms achieve an average convergence rate that is so far the fastest bounded convergence rate, among all SVM decomposition training algorithms to the best of our knowledge. The fast average convergence bound is achieved by a unique priority based sampling mechanism. (2) Unlike previous work (Provably fast training algorithm for support vector machines, 2001) the proposed algorithms work for general linear-nonseparable SVM and general non-linear SVR problems. This improvement is achieved by modeling new LP-type problems based on Karush–Kuhn–Tucker optimality conditions. (3) The proposed algorithms are the first parallel version of randomized sampling algorithms for SVM and SVR. Both the analytical convergence bound and the numerical results in a real application show that the proposed algorithm has good scalability. (4) We present demonstrations of the algorithms based on both synthetic data and data obtained from a real word application. Performance comparisons with SVM^light show that the proposed algorithms may be efficiently implemented.     

On tolerant fuzzy c-means clustering and tolerant possibilistic clustering

This paper presents two new types of clustering algorithms by using tolerance vector called tolerant fuzzy c-means clustering and tolerant possibilistic clustering. In the proposed algorithms, the new concept of tolerance vector plays very important role. The original concept is developed to handle data flexibly, that is, a tolerance vector attributes not only to each data but also each cluster. Using the new concept, we can consider the influence of clusters to each data by the tolerance. First, the new concept of tolerance is introduced into optimization problems. Second, the optimization problems with tolerance are solved by using Karush–Kuhn–Tucker conditions. Third, new clustering algorithms are constructed based on the optimal solutions for clustering. Finally, the effectiveness of the proposed algorithms is verified through numerical examples and its fuzzy classification function.     

Some principles of coordinated decision making in distributed systems

Conclusion The proposed approach leads to flexible decision support algorithms and procedures that easily adapt to changing requirements. The application of the proposed principles is illustrated in [12] with the object of allowing for the specific features of the problem and accelerating convergence of distributed decision support systems. The application of these principles to the construction of various control procedures and decision support scheme is demonstrated in [13–19]. At the present time, in connection with active transition to the market and operation in a rapidly changing reality, we can expect an increase in demand for algorithms, procedures, and schemes that divide the domains of competence, sharply delineate the domains of responsibility, and clearly separate the fields of action of the “center” and the “periphery” [11]. The need for such procedures will also be felt in financial management support [26–27] and in macro/micro economic modeling and forecasting [20–26]. This is due to the fact that in our rapidly changing world we are often unable to identify several separate criteria for optimization. We are often forced to look for a decision which is admissible by a whole range of formal and informal criteria and is stable under small perturbations of both the criteria and the external conditions [28–30]. Translated from Kibernetika i Sistemnyi Analiz, No. 2, pp. 161–175, March–April, 1996.     

Identification of the parameters of a convection diffusion system

Computational algorithms are proposed for the realization of gradient methods based on the solution of direct and conjugate problems in weak formulations for some complex inverse problems of the recovery of parameters of multicomponent parabolic distributed systems. The approach proposed makes it unnecessary to set up Lagrange functionals in explicit form and to use the Green function. Translated from Kibernetika i Sistemnyi Analiz, No. 1, pp. 42–63, January–February 2009.     

Construction algorithms and approximation bounds for the streaming cache placement problem in multicast networks

The streaming cache placement problem (SCPP) is considered. The SCPP is known to be NP-hard and MAX SNP-hard. It is shown that for the SCPP there is no approximation algorithm with a guarantee better than log k unless NP can be solved in sub-exponential time. Construction algorithms for the SCPP, based on two general techniques, are proposed. The results of computational experiments based on these two algorithms and their modifications are reported. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 6, pp. 124–137, November–December 2005.     

Solution of inverse boundary-value problems for multicomponent parabolic distributed systems

Computational algorithms implementing gradient methods based on solution of direct and adjoint problems in weak formulations are proposed for a number of complex-valued inverse problems of parameter renewal in multicomponent parabolic distributed systems. This approach makes it unnecessary to construct Lagrangian functionals explicitly and to use Green’s functions. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 4, pp. 49–73, July–August 2007.     

A mixed cellular method of matrix multiplication

A mixed cellular method of matrix multiplication is proposed that combines the Strassen method with a fast cellular method of matrix multiplication. The interaction of these methods makes it possible to decrease the multiplicative and additive complexities of well-known matrix multiplication algorithms by 25%. Estimates of computational complexity of cellular analogues of the mentioned algorithms are given. Translated from Kibernetika i Sistemnyi Analiz, No. 1, pp. 22–27, January–February 2009.     

Distinctive features of structural programming of synchronous sorting algorithms

Parallel (synchronous) algorithms of sorting arrays are considered. A modified synchronous sorting algorithm based on the pairwise exchange method is proposed and simulated. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 5, pp. 122–133, September–October 2006.     

Some stochastic Leontief-Ford models and their deterministic equivalents

Some stochastic Leontief-Ford models of ecological-economic interaction and their deterministic equivalents are proposed. A theory of stochastic variants of ecological-economic structural-type functions is developed. Appropriate algorithms are developed. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 3–10, May–June 2007.     

Nonnegative tensor factorization as an alternative Csiszar–Tusnady procedure: algorithms,convergence, probabilistic interpretations and novel probabilistic tensor latent variable analysis algorithms

In this paper we study Nonnegative Tensor Factorization (NTF) based on the Kullback–Leibler (KL) divergence as an alternative Csiszar–Tusnady procedure. We propose new update rules for the aforementioned divergence that are based on multiplicative update rules. The proposed algorithms are built on solid theoretical foundations that guarantee that the limit point of the iterative algorithm corresponds to a stationary solution of the optimization procedure. Moreover, we study the convergence properties of the optimization procedure and we present generalized pythagorean rules. Furthermore, we provide clear probabilistic interpretations of these algorithms. Finally, we discuss the connections between generalized Probabilistic Tensor Latent Variable Models (PTLVM) and NTF, proposing in that way algorithms for PTLVM for arbitrary multivariate probabilistic mass functions.     

Checking the satisfiability of formulas represented in disjunctive normal form in the language <Emphasis Type="Italic">L</Emphasis>. II

This paper concentrates on issues of implementation of the method proposed for checking the consistency of formulae of the language L and described in the first part of this work. Upper-bound estimates of time complexity of the corresponding algorithms are obtained. Part I of this article is published in No. 4 (2005). __________ Translated from Kibernetika i Sistemnyi Analiz, No. 6, pp. 11–19, November–December 2005.     

Structural models of algorithms in problems of applied programming. I. Formal algorithmic structures

A structural approach to modelling algorithms is proposed. The concept of an algorithmic structure with operations over algorithms is introduced. Models of algorithms constructed from algorithmic structures reflect aspects of algorithm representation and execution. Examples of structural and path models of algorithms are considered. Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 3–14, May-June 2009.     

Recognition of fractal graphs

Definitions of prefractal and fractal graphs are introduced, and they are used to formulate mathematical models in different fields of knowledge. The topicality of fractal-graph recognition from the point of view of fundamental improvement in the efficiency of the solution of algorithmic problems is emphasized. The class of polynomial algorithms for recognition of the prefractality of those graphs is proposed and rigorously substantiated. Translated from Kibernetika i Sistemnyi Analiz, No. 4, pp. 72–89, July–August, 1999.     

PID control of MIMO process based on rank niching genetic algorithm

Non-linear multiple-input multiple-output (MIMO) processes which are common in industrial plants are characterized by significant interactions and non- linearities among their variables. Thus, tuning several controllers in complex industrial plants is a challenge for process engineers and operators. An approach for adjusting the parameters of n proportional–integral–derivative (PID) controllers based on multiobjective optimization and genetic algorithms (GA) is presented in this paper. A modified genetic algorithm with elitist model and niching method is developed to guarantee a set of solutions (set of PID parameters) with different tradeoffs regarding the multiple requirements of the control performance. Experiments considering a fluid catalytic cracking (FCC) unit, under PI and dynamic matrix control (DMC) are carried out in order to evaluate the proposed method. The results show that the proposed approach is an alternative to classical techniques as Ziegler–Nichols rules and others.     

A new bootstrap-based forecast evaluation method tested on time series

This article proposes a new forecast quality evaluation method for parametric models. The method takes into account the nonequivalence of forecast errors induced by different accuracies of evaluation of unknown parameters at different time instants and uses nonuniform weighting. Optimal weights values are determined and several numerical algorithms are proposed for their approximate evaluation. The method is tested using time series that describe real economic processes. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 1, pp. 148–158, January–February 2008.