An efficient global optimization algorithm for multifactor dynamic systems

Abstract

In this study we present an efficient global optimization method, DIviding RECTangle (DIRECT) algorithm, for parametric analysis of dynamic systems. In a bounded constrained problem the DIRECT algorithm explores multiple potentially optimal subspaces in one search. The algorithm also eliminates the need for derivative calculations which are required in some efficient gradient‐based methods. In this study the first optimization example is to find the dynamic parameters of a tennis racket. The second example is a biomechanical parametric study of a heel‐toe running model governed by six factors. The effectiveness of the DIRECT algorithm is compared with a genetic algorithm in an analysis of heel‐toe running. The result shows that the DIRECT algorithm obtains an improved result in 83% less execution time. It is demonstrated that the straightforward DIRECT algorithm provides a general procedure for solving global optimization problems efficiently and confidently.     

Modeling an Augmented Lagrangian for Blackbox Constrained Optimization

Constrained blackbox optimization is a difficult problem, with most approaches coming from the mathematical programming literature. The statistical literature is sparse, especially in addressing problems with nontrivial constraints. This situation is unfortunate because statistical methods have many attractive properties: global scope, handling noisy objectives, sensitivity analysis, and so forth. To narrow that gap, we propose a combination of response surface modeling, expected improvement, and the augmented Lagrangian numerical optimization framework. This hybrid approach allows the statistical model to think globally and the augmented Lagrangian to act locally. We focus on problems where the constraints are the primary bottleneck, requiring expensive simulation to evaluate and substantial modeling effort to map out. In that context, our hybridization presents a simple yet effective solution that allows existing objective-oriented statistical approaches, like those based on Gaussian process surrogates and expected improvement heuristics, to be applied to the constrained setting with minor modification. This work is motivated by a challenging, real-data benchmark problem from hydrology where, even with a simple linear objective function, learning a nontrivial valid region complicates the search for a global minimum. Supplementary materials for this article are available online.     

Constrained optimization using a multipoint type chaotic Lagrangian method with a coupling structure

This article proposes a new constrained optimization method using a multipoint type chaotic Lagrangian method that utilizes chaotic search trajectories generated by Lagrangian gradient dynamics with a coupling structure. In the proposed method, multiple search points autonomously implement global search using the chaotic search trajectory generated by the coupled Lagrangian gradient dynamics. These points are advected to elite points (which are chosen by considering their objective function values and their feasibility) by the coupling in order to explore promising regions intensively. In this way, the proposed method successfully provides diversification and intensification for constrained optimization problems. The effectiveness of the proposed method is confirmed through application to various types of benchmark problem, including the coil spring design problem, the benchmark problems used in the special session on constrained real parameter optimization in CEC2006, and a high-dimensional and multi-peaked constrained optimization problem.     

A trust-region framework for constrained optimization using reduced order modeling

Reduced order models (ROM), such as proper orthogonal decomposition (POD), lead to powerful techniques to address computational challenges in PDE-constrained optimization. However, when incorporated within optimization strategies, ROMs are sufficiently accurate only in a restricted zone around the point in decision variable space where they are constructed. Consequently, the ROM needs to be updated in a systematic manner over the course of the optimization. As an enabling strategy, trust-region methods provide an excellent adaptive framework for ROM-based optimization. They not only restrict the optimization step within the ROM’s validity, but also synchronize ROM updates with information obtained during the course of optimization, thus providing a robust and globally convergent framework. This study extends the trust-region framework to constrained optimization problems, using two approaches. We first develop an exact penalty-based trust-region algorithm with correction schemes to ensure global convergence with ROM-based approximate models. Next, we develop a novel filter trust-region algorithm which utilizes refinement of the ROM, and a feasibility restoration phase. Both algorithms are applied to the optimization of a two-bed four-step isothermal pressure swing adsorption (PSA) system, for CO₂ capture. Although both algorithms converge to the same local optimum, the filter approach takes fewer trust-region iterations and requires less computational effort.     

Efficiently solving linear bilevel programming problems using off-the-shelf optimization software

Many optimization models in engineering are formulated as bilevel problems. Bilevel optimization problems are mathematical programs where a subset of variables is constrained to be an optimal solution of another mathematical program. Due to the lack of optimization software that can directly handle and solve bilevel problems, most existing solution methods reformulate the bilevel problem as a mathematical program with complementarity conditions (MPCC) by replacing the lower-level problem with its necessary and sufficient optimality conditions. MPCCs are single-level non-convex optimization problems that do not satisfy the standard constraint qualifications and therefore, nonlinear solvers may fail to provide even local optimal solutions. In this paper we propose a method that first solves iteratively a set of regularized MPCCs using an off-the-shelf nonlinear solver to find a local optimal solution. Local optimal information is then used to reduce the computational burden of solving the Fortuny-Amat reformulation of the MPCC to global optimality using off-the-shelf mixed-integer solvers. This method is tested using a wide range of randomly generated examples. The results show that our method outperforms existing general-purpose methods in terms of computational burden and global optimality.     

Finding all global optima of engineering design problems with discrete signomial terms

ABSTRACT

Many engineering design problems are frequently modelled as nonlinear programming problems with discrete signomial terms. In general, signomial programs are very difficult to solve for obtaining the globally optimal solution. This study reformulates the engineering design problem with discrete signomial terms as a mixed-integer linear program and finds all alternative global optima. Compared with existing exact methods, the proposed method uses fewer variables and constraints in the reformulated model and therefore efficiently solves the engineering problem to derive all global optima. Illustrative examples from the literature are solved to demonstrate the usefulness and efficiency of the proposed method.     

Case Studies in Optimization: Catenary Problem

This is the second paper in a series presenting case studies in modern large-scale constrained optimization 9 In this paper, we consider the shape of a hanging chain, which, in equilibrium, minimizes the potential energy of the chain. In addition to the tutorial aspects of this paper, we also emphasize the importance of certain modeling issues such as convex vs. nonconvex formulations of given problem. We will present several models of the problem and demonstrate differences in the number of iterations and solution time.     

Multi‐modal parameter identification by automata approach

Abstract

In this paper, we consider the multi‐modal function optimization problem. An automata model with improved learning schemes is proposed to solve the global optimization problem. Theoretically, we prove that the automaton converges to the global optimum with a probability arbitrarily close to 1. The numerical simulation results show that the automata approach is better than both the well‐known gradient approach and the simulated annealing method. The simulation results also show that our automata model converges faster than the other existing models in the literature.     

An efficient interior-point algorithm with new non-monotone line search filter method for nonlinear constrained programming

An efficient primal-dual interior-point algorithm using a new non-monotone line search filter method is presented for nonlinear constrained programming, which is widely applied in engineering optimization. The new non-monotone line search technique is introduced to lead to relaxed step acceptance conditions and improved convergence performance. It can also avoid the choice of the upper bound on the memory, which brings obvious disadvantages to traditional techniques. Under mild assumptions, the global convergence of the new non-monotone line search filter method is analysed, and fast local convergence is ensured by second order corrections. The proposed algorithm is applied to the classical alkylation process optimization problem and the results illustrate its effectiveness. Some comprehensive comparisons to existing methods are also presented.     

Bicriterial Design of Digital Filters

A problem of technical interest is the solution of approximation problems which make a tradeoff between the L ₂ norm and the L norm error criteria. This problem is investigated in the framework of filter design with respect to two conflicting optimality goals. The particular interest in L ₂-L norm compromise filters has been raised by a paper of Adams (IEEE Trans. on Circuits and Systems vol. 39, pp. 376–388, 1991), who suggested to compute such FIR filters by solution of certain constrained L ₂ approximation problems which require a proper choice of weights. It is shown in this paper that bicriterial filter design problems can be approached by classical methods from multicriteria optimization and that especially reference point approximation with the ideal point as reference point is a suitable tool to deal with Adams' problem. Solutions from this latter approach do especially not depend on the choice of weights and yield the best possible compromise filters with respect to a prescribed measure. The resulting optimization problems can be solved with (semi-infinite) programming methods having proven convergence under standard assumptions. Examples of L ₂-L norm compromise designs of a linear-phase FIR and an IIR filter are presented.     

求解约束优化问题的改进布谷鸟搜索算法

为了提高约束优化问题的求解精度和收敛速度,提出求解约束优化问题的改进布谷鸟搜索算法。首先分析了基本布谷鸟搜索算法全局搜索和局部搜索过程中的不足,对其中全局搜索和局部搜索迭代公式进行重新定义,然后以一定概率在最优解附近进行搜索。对12个标准约束优化问题和4个工程约束优化问题进行测试并与多种算法进行对比,实验结果和统计分析表明所提算法在求解约束优化问题上具有较强的优越性。     

Robust Parameter Estimation in Dynamic Systems

In this paper we present a practical method for robust parameter estimation in dynamic systems. In our study we follow the very successful approach for solving optimization problems in dynamic systems, namely the boundary value problem (BVP) approach. The suggested method combines multiple shooting for parameterizing dynamics, a flexible realization of the BVP principle, with a fast Gauss-Newton algorithm for solving the resulting constrained l ₁ problem. We give an overview of the theoretical background as well as the details of a numerical implementation. We discuss why the Gauss-Newton algorithm, which is known to perform well mainly on well-conditioned problems, is appropriate for parameter estimation problems, while quasi-Newton methods have only limited use for parameter estimation. The method is implemented on the basis of the direct multiple shooting method as implemented in PARFIT, thus inheriting all basic properties of PARFIT such as numerical stability, reliability and efficiency. The new code has been successfully applied to real-life parameter estimation problems in enzyme and chemical kinetics.     

A topological sorting algorithm for three‐layer channel routing

Abstract

Three‐layer channel routing is an important problem in VLSI layout. In this paper, we present a topological sorting algorithm to determine the topological order on nets depending on the operations of the vertical constraint graph (VCG) and the horizontal constraint graph (HCG). According to the order, the base router can finish the connections of all nets. In the algorithm, the HVH mode is assumed, and some examples including Deutsch's difficult example are used as test problems.     

Integrating a decomposition procedure with problem reduction for factory scheduling with disruptions: a simulation study

Dispatching rules are widely used in industry because schedules obtained from optimization procedures can be difficult to implement in the face of executional uncertainties. Barua et al. (Barua, A., Narasimhan, R., Upasani, A. and Uzsoy, R., Implementing global factory schedules in the face of stochastic disruptions. Int. J. Prod. Res., 2005, 43(4), 793–818) implement global schedules obtained from an optimization-based heuristic using a dispatching rule, and outperform myopic dispatching rules in the face of disruptions. However, the computation of the global schedules is still time-consuming for realistic instances. Upasani et al. (Upasani, A., Uzsoy, R. and Sourirajan, K., A problem reduction approach for scheduling semiconductor wafer fabrication facilities. IEEE Trans. Semicon. Manuf., 2006, 19, 216–225) develop a problem reduction scheme based on load disparity between work centres, and report significant reduction in CPU times with minimal loss of solution quality in deterministic experiments. In this paper we integrate the problem-reduction scheme to obtain global schedules with the dispatching approach of Barua et al. (Barua, A., Narasimhan, R., Upasani, A. and Uzsoy, R., Implementing global factory schedules in the face of stochastic disruptions. Int. J. Prod. Res., 2005, 43(4), 793–818) in a multi-product environment with stochastic machine breakdowns and job arrivals. A simulation model of a scaled-down wafer fabrication facility is used to evaluate the performance of the proposed procedures. Results show that the integrated procedure outperforms the benchmark dispatching rules while significantly reducing computation times.     

An approach to multi‐start clustering for global optimization with non‐linear constraints

This paper describes a multi‐start with clustering strategy for use on constrained optimization problems. It is based on the characteristics of non‐linear constrained global optimization problems and extends a strategy previously tested on unconstrained problems. Earlier studies of multi‐start with clustering found in the literature have focused on unconstrained problems with little attention to non‐linear constrained problems. In this study, variations of multi‐start with clustering are considered including a simulated annealing or random search procedure for sampling the design domain and a quadratic programming (QP) sub‐problem used in cluster formation. The strategies are evaluated by solving 18 non‐linear mathematical problems and six engineering design problems. Numerical results show that the solution of a one‐step QP sub‐problem helps predict possible regions of attraction of local minima and can enhance robustness and effectiveness in identifying local minima without sacrificing efficiency. In comparison to other multi‐start techniques found in the literature, the strategies of this study can be attractive in terms of the number of local searches performed, the number of minima found, whether the global minimum is located, and the number of the function evaluations required. Copyright © 2002 John Wiley & Sons, Ltd.     

Numerical methods for interactive multiple‐class image segmentation problems

In this article, we consider a bilaterally constrained optimization model arising from the semisupervised multiple‐class image segmentation problem. We prove that the solution of the corresponding unconstrained problem satisfies a discrete maximum principle. This implies that the bilateral constraints are satisfied automatically and that the solution is unique. Although the structure of the coefficient matrices arising from the optimality conditions of the segmentation problem is different for different input images, we show that they are M‐matrices in general. Therefore, we study several numerical methods for solving such linear systems and demonstrate that domain decomposition with block relaxation methods are quite effective and outperform other tested methods. We also carry out a numerical study of condition numbers on the effect of boundary conditions on the optimization problems, which provides some insights into the specification of boundary conditions as an input knowledge in the learning context. © 2010 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 20, 191–201, 2010     

A heuristic for optimizing stochastic activity networks with applications to statistical digital circuit sizing

A deterministic activity network (DAN) is a collection of activities, each with some duration, along with a set of precedence constraints, which specify that activities begin only when certain others have finished. One critical performance measure for an activity network is its makespan, which is the minimum time required to complete all activities. In a stochastic activity network (SAN), the durations of the activities and the makespan are random variables. The analysis of SANs is quite involved, but can be carried out numerically by Monte Carlo analysis. This paper concerns the optimization of a SAN, i.e., the choice of some design variables that affect the probability distributions of the activity durations. We concentrate on the problem of minimizing a quantile (e.g., 95%) of the makespan, subject to constraints on the variables. This problem has many applications, ranging from project management to digital integrated circuit (IC) sizing (the latter being our motivation). While there are effective methods for optimizing DANs, the SAN optimization problem is much more difficult; the few existing methods cannot handle large-scale problems. In this paper we introduce a heuristic method for approximately optimizing a SAN, by forming a related DAN optimization problem which includes extra margins in each of the activity durations to account for the variation. Since the method is based on optimizing a DAN, it readily handles large-scale problems. To assess the quality of the resulting suboptimal designs, we describe two widely applicable lower bounds on achievable performance in optimal SAN design. We demonstrate the method on a simplified statistical digital circuit sizing problem, in which the device widths affect both the mean and variance of the gate delays. Numerical experiments show that the resulting design is often substantially better than one in which the variation in delay is ignored, and is often quite close to the global optimum (as verified by the lower bounds).     

Elephant swarm water search algorithm for global optimization

The rising complexity of real-life optimization problems has constantly inspired computer researchers to develop new efficient optimization methods. Evolutionary computation and metaheuristics based on swarm intelligence are very popular nature-inspired optimization techniques. In this paper, the author has proposed a novel elephant swarm water search algorithm (ESWSA) inspired by the behaviour of social elephants, to solve different optimization problems. This algorithm is mainly based on the water search strategy of intelligent and social elephants during drought. Initially, we perform preliminary parametric sensitivity analysis for our proposed algorithm, developing guidelines for choosing the parameter values in real-life problems. In addition, the algorithm is evaluated against a number of widely used benchmark functions for global optimizations, and it is observed that the proposed algorithm has better performance for most of the cases compared with other state-of-the-art metaheuristics. Moreover, ESWSA performs better during fitness test, convergence test, computational complexity test, success rate test and scalability test for most of the benchmarks. Next, ESWSA is tested against two well-known constrained optimization problems, where ESWSA is found to be very efficient in term of execution speed and best fitness. As an application of ESWSA to real-life problem, it has been tested against a benchmark problem of computational biology, i.e., inference of Gene Regulatory Network based on Recurrent Neural Network. It has been observed that the proposed ESWSA is able to reach nearest to global minima and enabled inference of all true regulations of GRN correctly with less computational time compared with the other existing metaheuristics.     

Design of short fiber Bragg gratings by use of optimization

We demonstrate an optimization approach for designing fiber Bragg gratings. A layer-peeling inverse-scattering algorithm is used to produce an initial solution, which is optimized numerically with an iterative optimization method. To avoid problems with local minima, we use merit functions that are zero for wavelengths for which the predefined demands (acceptance limits) are fulfilled, making it possible to alter the local minima under the optimization process without disturbing the global minimum. Because short gratings are difficult to design with inverse scattering, and because the time consume of the optimization increases rapidly with the grating length, the method is particularly useful for designing short gratings. The method is also useful when the demands are complex and difficult to handle with inverse-scattering methods. Design examples are given, including a dispersionless bandpass filter suitable for dense wavelength-division multiplexing and a filter with linear reflectivity.     

Constrained multi-objective optimization algorithm with an ensemble of constraint handling methods

Different constraint handling techniques have been used with multi-objective evolutionary algorithms (MOEA) to solve constrained multi-objective optimization problems. It is impossible for a single constraint handling technique to outperform all other constraint handling techniques always on every problem irrespective of the exhaustiveness of the parameter tuning. To overcome this selection problem, an ensemble of constraint handling methods (ECHM) is used to tackle constrained multi-objective optimization problems. The ECHM is integrated with a multi-objective differential evolution (MODE) algorithm. The performance is compared between the ECHM and the same single constraint handling methods using the same MODE (using codes available from http://www3.ntu.edu.sg/home/EPNSugan/index.htm). The results show that ECHM overall outperforms the single constraint handling methods.