A framework for design optimization using surrogates

Design optimization is a computationally expensive process as it requires the assessment of numerous designs and each of such assessments may be based on expensive analyses (e.g. computational fluid dynamics method or finite element based method). One way to contain the computational time within affordable limits is to use computationally cheaper approximations (surrogates) in lieu of the actual analyses during the course of optimization. This article introduces a framework for design optimization using surrogates. The framework is built upon a stochastic, zero-order, population-based optimization algorithm, which is embedded with a modified elitism scheme, to ensure convergence in the actual function space. The accuracy of the surrogate model is maintained via periodic retraining and the number of data points required to create the surrogate model is identified by a k-means clustering algorithm. A comparison is provided between different surrogate models (Kriging, radial basis functions (Exact and Fixed) and Cokriging) using a number of mathematical test functions and engineering design optimization problems. The results clearly indicate that for a given fixed number of actual function evaluations, the surrogate assisted optimization model consistently performs better than a pure optimization model using actual function evaluations.     

A surrogate-model based multi-modal optimization algorithm

In this article a new algorithm for optimization of multi-modal, nonlinear, black-box objective functions is introduced. It extends the recently-introduced adaptive multi-modal optimization by incorporating surrogate modelling features similar to response surface methods. The resulting algorithm has reduced computational intensity and is well-suited for optimization of expensive objective functions. It relies on an adaptive, multi-resolution mesh to obtain an initial estimation of the objective function surface. Local surrogate models are then constructed and used to generate additional trial points around the local minima discovered. The steps of mesh refinement and surrogate modelling continue until convergence is achieved. The algorithm produces progressively accurate surrogate models, which can be used for post-optimization studies such as sensitivity and tolerance analyses with minimal computational effort. This article demonstrates the effectiveness of the algorithm using comparative optimization of several multi-modal objective functions, and shows an engineering application of the design of a power electronic converter.     

Locally weighted regression models for surrogate-assisted design optimization

We consider engineering design optimization problems where the objective and/or constraint functions are evaluated by means of computationally expensive blackboxes. Our practical optimization strategy consists of solving surrogate optimization problems in the search step of the mesh adaptive direct search algorithm. In this paper, we consider locally weighted regression models to build the necessary surrogates, and present three ideas for appropriate and effective use of locally weighted scatterplot smoothing (LOWESS) models for surrogate optimization. First, a method is proposed to reduce the computational cost of LOWESS models. Second, a local scaling coefficient is introduced to adapt LOWESS models to the density of neighboring points while retaining smoothness. Finally, an appropriate order error metric is used to select the optimal shape coefficient of the LOWESS model. Our surrogate-assisted optimization approach utilizes LOWESS models to both generate and rank promising candidates found in the search and poll steps. The “real” blackbox functions that govern the original optimization problem are then evaluated at these ranked candidates with an opportunistic strategy, reducing CPU time significantly. Computational results are reported for four engineering design problems with up to six variables and six constraints. The results demonstrate the effectiveness of the LOWESS models as well as the order error metric for surrogate optimization.     

MISO: mixed-integer surrogate optimization framework

We introduce MISO, the mixed-integer surrogate optimization framework. MISO aims at solving computationally expensive black-box optimization problems with mixed-integer variables. This type of optimization problem is encountered in many applications for which time consuming simulation codes must be run in order to obtain an objective function value. Examples include optimal reliability design and structural optimization. A single objective function evaluation may take from several minutes to hours or even days. Thus, only very few objective function evaluations are allowable during the optimization. The development of algorithms for this type of optimization problems has, however, rarely been addressed in the literature. Because the objective function is black-box, derivatives are not available and numerically approximating the derivatives requires a prohibitively large number of function evaluations. Therefore, we use computationally cheap surrogate models to approximate the expensive objective function and to decide at which points in the variable domain the expensive objective function should be evaluated. We develop a general surrogate model framework and show how sampling strategies of well-known surrogate model algorithms for continuous optimization can be modified for mixed-integer variables. We introduce two new algorithms that combine different sampling strategies and local search to obtain high-accuracy solutions. We compare MISO in numerical experiments to a genetic algorithm, NOMAD version 3.6.2, and SO-MI. The results show that MISO is in general more efficient than NOMAD and the genetic algorithm with respect to finding improved solutions within a limited budget of allowable evaluations. The performance of MISO depends on the chosen sampling strategy. The MISO algorithm that combines a coordinate perturbation search with a target value strategy and a local search performs best among all algorithms.     

Kriging Surrogate Based Multi-objective Optimization of Bulk Vinyl Acetate Polymerization with Branching

Despite the established superiority in finding the global as well as well-spread Pareto optimal (PO) points, the need of more numbers of function evaluations for population based evolutionary optimization techniques leads to a computationally demanding proposal. The case becomes more miserable if the function evaluations are carried out using a first principle based computationally expensive model, making the proposal not fit for online usage of the application. In this work, a Kriging based surrogate model has been proposed to replace a computationally expensive model to save execution time while performing an optimization task. A multi-objective optimization study has been carried out for the bulk vinyl acetate polymerization with long-chain branching using these surrogate as well as expensive models and Kriging PO solutions similar to those found by the first principle models are obtained with a close to 85% savings in function evaluations.     

Evolutionary reliable regional Kriging surrogate for expensive optimization

This work proposes a sequential optimization algorithm, EORKS, combining a Kriging surrogate from an adaptive sampling and an iterative constrained search in the dynamic reliable regions to reduce the sampling size in expensive optimization. A surrogate established from small samples is liable to limited generality, which leads to a false prediction of optimum. EORKS applies Kriging variance to establish the reliable region neighbouring the learning samples to constrain the evolutionary searches of the surrogate. The verified quasi-optimum is used as an additional sample to dynamically update the regional model according to the prediction accuracy. A hybrid infilling strategy switches between the iterative quasi-optima and the maximum expected improvement from Kriging to prevent early convergence of local optimum. EORKS provides superior optima in several benchmark functions and an engineering design problem, using much smaller samples compared with the literature results, which demonstrates the sampling efficiency and searching robustness.     

Infill sampling criteria for surrogate-based optimization with constraint handling

This article discusses the benefits of different infill sampling criteria used in surrogate-based constrained global optimization. A new method which selects multiple updates based on Pareto optimal solutions is introduced showing improvements over a number of existing methods. The construction of surrogates (also known as meta-models or response surface models) involves the selection of a limited number of designs which are analysed using the original expensive functions. A typical approach involves two stages. First the surrogate is built using an initial sampling plan; the second stage updates the model using an infill sampling criterion to select further designs that offer improvement. Selecting multiple update points at each iteration, allowing distribution of the expensive function evaluations on several processors offers large potential for accelerating the overall optimization process. This article provides a comparison between different infill sampling criteria suitable for selecting multiple update points in the presence of constraints.     

Applicability of surrogates to improve efficiency of particle swarm optimization for simulation-based problems

Particle swarm optimization (PSO) is a population-based, heuristic technique based on social behaviour that performs well on a variety of problems including those with non-convex, non-smooth objective functions with multiple minima. However, the method can be computationally expensive in that a large number of function calls is required. This is a drawback when evaluations depend on an off-the-shelf simulation program, which is often the case in engineering applications. An algorithm is proposed which incorporates surrogates as a stand-in for the expensive objective function, within the PSO framework. Numerical results are presented on standard benchmarking problems and a simulation-based hydrology application to show that this hybrid can improve efficiency. A comparison is made between the application of a global PSO and a standard PSO to the same formulations with surrogates. Finally, data profiles, probability of success, and a measure of the signal-to-noise ratio of the the objective function are used to assess the use of a surrogate.     

Combinatorial optimization of weld sequence by using a surrogate model to mitigate a weld distortion

Choosing an optimal sequence from the set of all possible combinations of a weld’s sub-passes is always a challenge for designers. The solution of such combinatorial optimization problems is limited by the available resources. For example, having n sub-passes leads to choosing from 2 ⁿ  × n! possible combinations of the sub-passes, e.g., 46,080 for n = 6. It is not feasible to choose the optimal sequence by evaluating all possible combinations either experimentally or by simulation models. The purpose of using a surrogate model based on a simulation model is to find the solution in the space of all possible combinations with a significant decrease in computational expenses. In effect, the surrogate model constructs an approximation model from some combinations of solutions of a more expensive model to mimic the behavior of the simulation model as closely as possible but at a much lower computational cost. This surrogate model, then, could be used to approximate the behavior of the other combinations. In this paper, a surrogate model is demonstrated that minimizes the distortion in a pipe girth weld with six sub-passes by analyzing only 14 combinations of sub-passes from total of 48 possible combinations. A comparison between the results of the surrogate model and the full transient FEM analysis all possible combinations shows the accuracy of the algorithm/model.     

Reliability-based design optimization with progressive surrogate models

Reliability-based design optimization (RBDO) has traditionally been solved as a nested (bilevel) optimization problem, which is a computationally expensive approach. Unilevel and decoupled approaches for solving the RBDO problem have also been suggested in the past to improve the computational efficiency. However, these approaches also require a large number of response evaluations during optimization. To alleviate the computational burden, surrogate models have been used for reliability evaluation. These approaches involve construction of surrogate models for the reliability computation at each point visited by the optimizer in the design variable space. In this article, a novel approach to solving the RBDO problem is proposed based on a progressive sensitivity surrogate model. The sensitivity surrogate models are built in the design variable space outside the optimization loop using the kriging method or the moving least squares (MLS) method based on sample points generated from low-discrepancy sampling (LDS) to estimate the most probable point of failure (MPP). During the iterative deterministic optimization, the MPP is estimated from the surrogate model for each design point visited by the optimizer. The surrogate sensitivity model is also progressively updated for each new iteration of deterministic optimization by adding new points and their responses. Four example problems are presented showing the relative merits of the kriging and MLS approaches and the overall accuracy and improved efficiency of the proposed approach.     

A Constraint Mapping Approach to the Structural Optimization of an Expensive Model using Surrogates

The use of response surface methods are well established in the global optimization of expensive functions, the response surface acting as a surrogate to the expensive function objective.In structural design however, the change in objective may vary little between the two models: it is more often the constraints that change with models of varying fidelity. Here approaches are described whereby the coarse model constraints are mapped so that the mapped constraints more faithfully approximate the fine model constraints. The shape optimization of a simple structure demonstrates the approach.     

A mixed-integer simulation-based optimization approach with surrogate functions in water resources management

Efficient and powerful methods are needed to overcome the inherent difficulties in the numerical solution of many simulation-based engineering design problems. Typically, expensive simulation codes are included as black-box function generators; therefore, gradient information that is required by mathematical optimization methods is entirely unavailable. Furthermore, the simulation code may contain iterative or heuristic methods, low-order approximations of tabular data, or other numerical methods which contribute noise to the objective function. This further rules out the application of Newton-type or other gradient-based methods that use traditional finite difference approximations. In addition, if the optimization formulation includes integer variables the complexity grows even further. In this paper we consider three different modeling approaches for a mixed-integer nonlinear optimization problem taken from a set of water resources benchmarking problems. Within this context, we compare the performance of a genetic algorithm, the implicit filtering algorithm, and a branch-and-bound approach that uses sequential surrogate functions. We show that the surrogate approach can greatly improve computational efficiency while locating a comparable, sometimes better, design point than the other approaches.     

A surrogate assisted parallel multiobjective evolutionary algorithm for robust engineering design

A number of multi-objective evolutionary algorithms have been proposed in recent years and many of them have been used to solve engineering design optimization problems. However, designs need to be robust for real-life implementation, i.e. performance should not degrade substantially under expected variations in the variable values or operating conditions. Solutions of constrained robust design optimization problems should not be too close to the constraint boundaries so that they remain feasible under expected variations. A robust design optimization problem is far more computationally expensive than a design optimization problem as neighbourhood assessments of every solution are required to compute the performance variance and to ensure neighbourhood feasibility. A framework for robust design optimization using a surrogate model for neighbourhood assessments is introduced in this article. The robust design optimization problem is modelled as a multi-objective optimization problem with the aim of simultaneously maximizing performance and minimizing performance variance. A modified constraint-handling scheme is implemented to deal with neighbourhood feasibility. A radial basis function (RBF) network is used as a surrogate model and the accuracy of this model is maintained via periodic retraining. In addition to using surrogates to reduce computational time, the algorithm has been implemented on multiple processors using a master–slave topology. The preliminary results of two constrained robust design optimization problems indicate that substantial savings in the actual number of function evaluations are possible while maintaining an acceptable level of solution quality.     

An adaptive metamodel-based global optimization algorithm for black-box type problems

In this article, an adaptive metamodel-based global optimization (AMGO) algorithm is presented to solve unconstrained black-box problems. In the AMGO algorithm, a type of hybrid model composed of kriging and augmented radial basis function (RBF) is used as the surrogate model. The weight factors of hybrid model are adaptively selected in the optimization process. To balance the local and global search, a sub-optimization problem is constructed during each iteration to determine the new iterative points. As numerical experiments, six standard two-dimensional test functions are selected to show the distributions of iterative points. The AMGO algorithm is also tested on seven well-known benchmark optimization problems and contrasted with three representative metamodel-based optimization methods: efficient global optimization (EGO), GutmannRBF and hybrid and adaptive metamodel (HAM). The test results demonstrate the efficiency and robustness of the proposed method. The AMGO algorithm is finally applied to the structural design of the import and export chamber of a cycloid gear pump, achieving satisfactory results.     

基于统计提升准则的注塑工艺参数优化

为了同时改善生产平板型注塑制品时的总体收缩度和收缩均匀度,提出基于统计提升准则的注塑成型工艺参数的多目标优化方法,寻找平衡两个质量指标的优化设计.首先利用小规模的实验设计方法获得建模数据集,针对应用中存在的建模数据奇异点问题提出一种数据预处理方法,并依此分别建立两个指标的初始替代模型,用于代替优化过程中代价高昂的计算分析;随后依据Pareto统计提升准则寻找新的采样点加入建模数据集来重新建模,使寻优结果不断趋近真实的Pareto前沿.仿真结果表明,较常规的建模优化方法,本文提出的方法能使用较少的采样数据,显著地改善平板制品的收缩质量.对于HDPE材质的矩形制品,保压曲线先恒定后线性递减可以获得好的收缩均匀度,使用压力上限值恒定保压可以获得好的平均收缩度.     

基于全局灵敏度分析的某自动装填机构轻量化设计

针对某自动装填机构轻量化设计中出现参数多、模型计算量大等问题,提出将全局灵敏度分析与代理模型技术相结合的优化策略。通过基于Morris轨迹的全局灵敏度分析从32个系统参数中确定14个关键参数,基于拉丁超立方采样技术及径向基函数神经网络技术(Radial basis function neural networks, RBF NN)建立系统响应关于关键参数的代理模型,用多岛遗传算法对系统参数进行优化求解,致机构重量下降21.8%。数值检验结果表明仅含关键参数的代理模型预测精度较高,证明该方法在多参数复杂系统结构轻量化设计中的有效性。     

A surrogate-assisted particle swarm optimization algorithm based on efficient global optimization for expensive black-box problems

Evolutionary algorithms cannot effectively handle computationally expensive problems because of the unaffordable computational cost brought by a large number of fitness evaluations. Therefore, surrogates are widely used to assist evolutionary algorithms in solving these problems. This article proposes an improved surrogate-assisted particle swarm optimization (ISAPSO) algorithm, in which a hybrid particle swarm optimization (PSO) is combined with global and local surrogates. The global surrogate is not only used to predict fitness values for reducing computational burden but also regarded as a global searcher to speed up the global search process of PSO by using an efficient global optimization algorithm, while the local one is constructed for a local search in the neighbourhood of the current optimal solution by finding the predicted optimal solution of the local surrogate. Empirical studies on 10 widely used benchmark problems and a real-world structural design optimization problem of a driving axle show that the ISAPSO algorithm is effective and highly competitive.     

Efficient global optimization for high-dimensional constrained problems by using the Kriging models combined with the partial least squares method

In many engineering optimization problems, the number of function evaluations is often very limited because of the computational cost to run one high-fidelity numerical simulation. Using a classic optimization algorithm, such as a derivative-based algorithm or an evolutionary algorithm, directly on a computational model is not suitable in this case. A common approach to addressing this challenge is to use black-box surrogate modelling techniques. The most popular surrogate-based optimization algorithm is the efficient global optimization (EGO) algorithm, which is an iterative sampling algorithm that adds one (or many) point(s) per iteration. This algorithm is often based on an infill sampling criterion, called expected improvement, which represents a trade-off between promising and uncertain areas. Many studies have shown the efficiency of EGO, particularly when the number of input variables is relatively low. However, its performance on high-dimensional problems is still poor since the Kriging models used are time-consuming to build. To deal with this issue, this article introduces a surrogate-based optimization method that is suited to high-dimensional problems. The method first uses the ‘locating the regional extreme’ criterion, which incorporates minimizing the surrogate model while also maximizing the expected improvement criterion. Then, it replaces the Kriging models by the KPLS(+K) models (Kriging combined with the partial least squares method), which are more suitable for high-dimensional problems. Finally, the proposed approach is validated by a comparison with alternative methods existing in the literature on some analytical functions and on 12-dimensional and 50-dimensional instances of the benchmark automotive problem ‘MOPTA08’.     

Update-based evolution control: A new fitness approximation method for evolutionary algorithms

Evolutionary algorithms are robust optimization methods that have been used in many engineering applications. However, real-world fitness evaluations can be computationally expensive, so it may be necessary to estimate the fitness with an approximate model. This article reviews design and analysis of computer experiments (DACE) as an approximation method that combines a global polynomial with a local Gaussian model to estimate continuous fitness functions. The article incorporates DACE in various evolutionary algorithms, to test unconstrained and constrained benchmarks, both with and without fitness function evaluation noise. The article also introduces a new evolution control strategy called update-based control that estimates the fitness of certain individuals of each generation based on the exact fitness values of other individuals during that same generation. The results show that update-based evolution control outperforms other strategies on noise-free, noisy, constrained and unconstrained benchmarks. The results also show that update-based evolution control can compensate for fitness evaluation noise.     

Noise-resilient estimation of optical flow by use of overlapped basis functions

Conventional techniques for the computation of optical flow from image gradients are used to formulate the problem as a nonlinear optimization that comprises a gradient constraint term and a field smoothness factor. The results of these techniques are often erroneous, highly sensitive to noise and numerical precision, determined sparsely, and computationally expensive. We regularize the gradient constraint equation by modeling optical flow as a linear combination of a set of overlapped basis functions. We develop a theory for estimating model parameters robustly and reliably. We prove that the extended-least-squares solution proposed here is unbiased and robust to small perturbations in the gradient estimates and to mild deviations from the gradient constraint. Our solution is obtained with a numerically stable sparse matrix inversion, which gives a reliable flow-field estimate over the entire frame. To validate our claims, we perform a series of experiments on standard benchmark data sets at a range of noise levels. Overall, our algorithm outperforms by a wide margin the others considered in the comparison. We demonstrate the applicability of our algorithm to image mosaicking and to motion superresolution through experiments on noisy compressed sequences. We conclude that our flow-field model offers greater accuracy and robustness than conventional optical flow techniques in a variety of situations and permits real-time operation.